Torne River Junosuando Sweden

Disable Billing for Google Cloud Project when billing exceeds the budget limit

In this article, I will explain how you can create a Google Cloud Platform (GCP) Cloud Function that you can use to cap the cost if the cost is higher than the budget the billing will be disabled.

You may ask why do I want to do this; you can set up billing notifications in GCP that send you an email when your budget limits are achieved, yes that works well sometimes. There are times when you may have installed an update to the Cloud Engine App that occurs on the network edge, starting to generate a large amount of traffic, and your billing increases dramatically.

I had an issue once when I wrote a Cloud Function. I published it, and there was something wrong in the code within minutes the incorrect code had generated over 100 dollars to my billing in seconds, things happen it’s better to be sorry and to play it safe than just go in a wimp till you get a bill and you say what did happen?

Google was very good with my Cloud Function that went down, I contacted their billing department and explained what happened, and they paid back the money. If it happens again, I now have a Cloud Function that will stop the billing and limit the cost.

The GCP Architect Services

In the GCP I’m using the following services as show in the diagram below:

Disabling Billing

If the total spend of a project exceeds the budget fixed for the project, the Billing will be disabled for the project using Cloud Billing API and also send a notification. One word about disabling Billing through the Google Billing API, when you programmatically send a diable billing through the BIlling API, your billing account will go from active to inactive. It will shut down any services that you have on the GCP project. Say you are running your WordPress site on the Google Compute Engine, your WordPress site will go down, be careful when using the Cloud Function to disable Billing.

Let’s get started

Let’s get started to set up all the GCP services to tie all the pieces together so that you will be able to disable billing for your GCP project.

Billing Notifications

The first thing we need to do is to create a budget alert for our GCP Billing, in the Google Cloud Platform Console navigate to Billing -> Budgets & alerts.

Budget Alerts
Budget Alerts

Create an alert use the project id as the name of your billing alert and select the alert to be sent to Pub/Sub – named the Sub/Pub topic to budgets-notifications.

When you save the billing alert you would see something like this.

Budget Alert
Budget Alert

Now you Billing Alert is setup, lets move on.

Cloud Pub/Sub

When you created a budget alert, you had that the option to create a Pub/Sub topic – you can now navigate to Pub/Sub and look at your topic and to verify after you have published your Cloud Function that the Cloud Function is subscribed to your Pub/Sub topic.

Enable Cloud Billing API

For the Cloud Function to be able to disable billing, we need to enable the Cloud Billing API, navigate to Google Cloud Platform Console APIs & Services, and enable the Cloud Billing API.

CloudBilling
CloudBilling

Secret Manager

I store all my secrets for the Cloud Function with the Secret Manager. You can read how to use the Secret Manager in this article Using Secret Manager in a Google Cloud Function with Python.

Creating Cloud Function with Python

Now you have the Secret Manager for Python installed in your environment, next you have to create your requirements.txt. You need this when you publish your Cloud Function to GCP.

google-api-python-client==1.7.4
oauth2client==4.1.3
google-cloud-secret-manager==1.0.0

I’m using three different Python libraries for the Cloud Function. I’m using the oauth2client to authenticate the function with GCP so I can use the google-api-python-client to disable the Billing API. The google-cloud-secret-manager I’m using to interact with the secret manager.

The Cloud Function Code in Python

Below is the code for the cloud function, I’m using the Secret Manager to manage my Slack Hook Key, the mailgun credentials as I using Slack and Mailgun to send me a notification at the same time as I disable the billing account, hopefully, I will catch the notification on these mediums in time, so I can analyze why the billing was shutdown and enable billing if required. I also use Environment variables to set my Environment variables.

Cloud Function Environment Variable
Cloud Function Environment Variable

I’m not going to explain all the lines of code, basically, we have a Pub/Sub trigger Cloud Function, when data starts flowing from billing it will trigger the function to execute, the cloud function just lives for a short time. The payload that comes from the Pub/Sub topic looks like this.

{'budgetDisplayName': 'torbjorn-zetterlund', 'costAmount': 0.59, 'costIntervalStart': '2020-06-01T07:00:00Z', 'budgetAmount': 50.0, 'budgetAmountType': 'SPECIFIED_AMOUNT', 'currencyCode': 'EUR'}

The Cloud Functions read this data and make a determination if the costAmount is higher than the budgetAmount, if that is the case it will trigger the disabling of the cloud billing and send email notification via mailgun and Slack notification to the Slack webhook. Below is the full code snippet.

You can download the Cloud Function from GitHub.
import logging
import os
import requests

# Install Google Libraries
from google.cloud import secretmanager

# Setup the Secret manager Client
client = secretmanager.SecretManagerServiceClient()
# Get the sites environment credentials
project_id = os.environ["PROJECT_NAME"]

# Get the secret for Slack
secret_name = "slack-hook-key"
resource_name = f"projects/{project_id}/secrets/{secret_name}/versions/latest"
response = client.access_secret_version(resource_name)
slackhookkeyurl = response.payload.data.decode('UTF-8')

# Get the secret for Mailgun
secret_name = "mailgun_domain"
resource_name = f"projects/{project_id}/secrets/{secret_name}/versions/latest"
response = client.access_secret_version(resource_name)
mailgundomain = response.payload.data.decode('UTF-8')

# Get the secret for Mailgun
secret_name = "mailgun-api"
resource_name = f"projects/{project_id}/secrets/{secret_name}/versions/latest"
response = client.access_secret_version(resource_name)
mailgunapi = response.payload.data.decode('UTF-8')

# Request Header
headers = {
    'Content-Type': 'application/json'
}

def gpi_techlab_budgets_notifications(data, context):

    import base64
    import json
    pubsub_budget_notification_data = json.loads(base64.b64decode(data['data']).decode('utf-8'))
    logging.info('Logging Details: {}'.format(pubsub_budget_notification_data))

    # The budget alert name must be created with the project_id you want to cap
    budget_project_id = pubsub_budget_notification_data['budgetDisplayName']

    # The budget amount to cap costs
    budget = pubsub_budget_notification_data['budgetAmount']

    logging.info('Handling budget notification for project id: {}'.format(budget_project_id))
    logging.info('The budget is set to {}'.format(budget))
    logging.info('Handling the cost amount of : {} for the budget notification period / month, or technically '
                 'the costIntervalStart : {}'.format(pubsub_budget_notification_data['costAmount'],
                                                     pubsub_budget_notification_data['costIntervalStart']))

    # If the billing is already disabled, stop Cloud Function execution
    if not __is_billing_enabled(budget_project_id):
        raise RuntimeError('Billing already in disabled state')
    else:
        # get total costs accrued per budget alert period

        # Calculating the total as the sum of the cost amounts which are values to start intervals keys in the dict
        monthtotal = pubsub_budget_notification_data['costAmount']
        logging.info('The month to date total is {}'.format(monthtotal))

        # Disable or not the billing for the project id depending on the total and the budget
        if monthtotal < budget:
            logging.info('No action will be taken on the total amount of {} for the period {} .'
                        .format(monthtotal, pubsub_budget_notification_data['costIntervalStart']))
        else:
            logging.info('The monthly total is more than {} euros for period {}, the monthly budget amount is set to {}, the billing will be disable for project id {}.'
                        .format(budget, pubsub_budget_notification_data['costIntervalStart'], monthtotal, budget_project_id))
            payload = '{{"text":"The monthly total is more than {} euros for period {}, the monthly budget amount is set to {}, the billing will be disable for project id {}."}}'\
                        .format(monthtotal, pubsub_budget_notification_data['costIntervalStart'], budget, budget_project_id)
            response = requests.request("POST", slackhookkeyurl, headers=headers, data=payload)
            emailtext = 'The monthly total is more than {} euros for period {}, the monthly budget amount is set to {}, the billing will be disable for project id {}.'\
                        .format(monthtotal, pubsub_budget_notification_data['costIntervalStart'],  budget, budget_project_id)
            __send_email_message_mailgun(emailtext)
            #__disable_billing_for_project(budget_project_id)
            
def __is_billing_enabled(project_id):

    service = __get_cloud_billing_service()
    billing_info = service.projects().getBillingInfo(name='projects/{}'.format(project_id)).execute()
    if not billing_info or 'billingEnabled' not in billing_info:
        return False
    return billing_info['billingEnabled']

def __get_cloud_billing_service():
    # Creating credentials to be used for authentication, by using the Application Default Credentials
    # The credentials are created  for cloud-billing scope
    from oauth2client.client import GoogleCredentials
    credentials = GoogleCredentials.get_application_default()

    # The name and the version of the API to use can be found here https://developers.google.com/api-client-library/python/apis/
    from apiclient import discovery
    return discovery.build('cloudbilling', 'v1', credentials=credentials, cache_discovery=False)

def __disable_billing_for_project(project_id):

    service = __get_cloud_billing_service()
    billing_info = service.projects()\
        .updateBillingInfo(name='projects/{}'.format(project_id), body={'billingAccountName': ''}).execute()
    assert 'billingAccountName' not in billing_info

def __send_email_message_mailgun(emailtext):
	return requests.post(
        "https://api.mailgun.net/v3/" + mailgundomain + "/messages",
        auth=("api", mailgunapi),
        data={"from": "Excited User <[email protected]>",
                "to": "[email protected]",
                "subject": "Budget Notification",
                "text": emailtext})

Authorization for the Cloud Function

At runtime, the Cloud Function uses a service account, which is listed at the bottom of the Cloud Function details page in the Google Cloud Platform Console.

You can manage Billing Admin permissions on the Google Cloud Platform Console Billing page. Add the Cloud Function runtime service account as a member and give it Billing Account Administrator privileges.

To give the Cloud Function runtime service account the Browser role, you need to add it as a member to each project to monitor through the Google Cloud Platform IAM page and give it the Browser role.

Cost

The Cloud Billing API, Cloud Pub/Sub and Cloud Function call and data are on the free tier.

Wrapping up

I hope I have been clear in how you can set up a Cloud Function to disable your GCP project billing if you have any questions please comment below and I will provide answers to the best of my knowledge.


Posted

in

, , ,

by

Comments

8 responses to “Disable Billing for Google Cloud Project when billing exceeds the budget limit”

  1. […] If you want to be able to control your Google Cloud Platform cost, this is a great article to read – Disable Billing for Google Cloud Project when billing exceeds the budget limit […]

  2. Temmame Said Avatar
    Temmame Said

    Hello,

    Nice article, unfortunately the Pub/Sub topic part is not very clear.
    Can you give some details about the topic budgets-notifications and how you did the subscruption ?

    Thank you

    1. Torbjorn Zetterlund Avatar
      Torbjorn Zetterlund

      In the article, I explain how you from the billing function in the Google Cloud publish the billing data to the pub/sub topic. The Cloud Function is the subscriber to the billing data topic in pub/sub. When the Cloud Function is first deployed, it is deployed with the pub/sub trigger, that is how the Cloud Function becomes a subscriber.

      Hope that was clearly describing the relationships between publisher and subscriber.

      1. Tammem Said Avatar
        Tammem Said

        Yes actually I didn’t see that I can switch the trigger HTTP to Cloud Pub/Sub while creating the Cloud function, now it’s more clear. Thank you very much for your answer.

  3. Bapi Mohanty Avatar
    Bapi Mohanty

    If you disable billing for the project, the project will be deleted. How can the project be enabled again?

    1. torbjorn Avatar

      You have 30 days to enable it again after it has been disabled. You can find more details here – https://cloud.google.com/billing/docs/how-to/modify-project

      To summaries:
      Enable billing for an existing project
      To use Google Cloud resources in a project, billing must be enabled on the project. Billing is enabled when the project is linked to an active Cloud Billing account. Billing can become disabled on a project for one of the following reasons:

      The project is unlinked from its billing account, disabling billing on the project.
      The Cloud Billing account that is linked to the project is closed or suspended.
      The project was shut down (deleted) and then restored within the 30-day recovery period.

      If you have a project where billing has been disabled, all billable services within that project are stopped. Re-enable billing on the project to restart the resources.

      After billing is re-enabled, it might take up to 24 hours for the resources to start up again.
      Some services might need to be restarted manually. For more information, see Restarting Google Cloud Services.
      While billing is disabled on a project, some resources might be deleted and might not be fully recoverable. Learn about data deletion on Google Cloud.

  4. Divya Avatar
    Divya

    Is this python code still valid ?

Leave a Reply

Your email address will not be published. Required fields are marked *