Using Cloud Build as your CI/CO for multiple Cloud Functions

Norway ice fishers Tromso
Norway ice fishers Tromso

There are many CI/CO tools available in the market today like GitLab CI, CircleCI I’m not so much in favor of these CI/CO services even if they have some cool features. I have been a Google Cloud Platform user for a few years now, and I prefer to use the tools that are available to me by a cloud platform that I use.

In my case, for CI/CO I’m using Google Cloud Build that is available in the Google Cloud Platform, in this how-to I will explain how I use Google Cloud Build to automatically deploy cloud functions to Google Cloud Function when an update is pushed to a GitHub repo.

Cloud Functions

I created a service to identify and track brand name being used and sold by illegal merchants on online platforms, for this I had put together a cloud architecture on the Google Cloud Platform that used serverless computing. I created a number of cloud functions Bots that run on the Google Cloud Function platform. The architecture looked like this.

As I had more than one cloud function, I wanted to keep all the cloud functions in one GitHub repository, and only the cloud function that was changed would be deployed by a trigger in the Cloud Build.

Organize cloud functions on GitHub

The way I organized the GitHub repo was by creating sub-folders for each of my cloud functions. It looked like this:

github sub-directories
github sub-directories

In each of the sub-directories, I created a cloudbuild.yaml file that has the instructions for Cloud Build on how to build and deploy. The cloudbuild.yaml looks like this.

steps:
  - name: "gcr.io/cloud-builders/gcloud"
    waitFor: ["-"]
    args:
      - functions
      - deploy
      - googlecloudbot_http
      - --runtime=python37
      - --trigger-http
      - --region=europe-west1
      - --memory=128MB
    id: "deploying-a-serverless-function"
    dir: "gpi-techlab-googlesearchbot"

Google Cloud Build

In the Google Cloud Platform Console, I selected Cloud Build. What I needed to do is to create a trigger that starts the build action to deploy a new version. I choose to Push to a branch as the Event for the main branch, in my case Master. This would be different depending on the size of the team working with the branch, in my case, it’s just me. I would suggest to Push a new tag if a team is working on the GitHub branch.

Google cloud build trigger
Google cloud build trigger

I created a trigger for each functions sub-directories in the GitHub branch, to only make that sub-directory to deploy I used cloudbuild.yaml inside each GitHub branch sub-directory. If the trigger was to detect a code change on the GitHub branch, I wanted it only to be trigger on the sub-directory that the code change to start the build process.

I achieved that by creating what include files filter I wanted, I specified the include file filter with the sub-directory – as shown in the example below I specified gpi-techlab-bingboot/** that would only include all files from the sub-directory gpi-techlab-bingboot

And I also specified that I had a .yaml file and that the .yaml file could be found in the sub-directory gpi-techlab-bingboot

Conclusion

That is all for this how-to, it is straight forward, and it works excellent. If you have questions you can contact me or you can read more about Google Cloud Build triggers here.

6 comments On Using Cloud Build as your CI/CO for multiple Cloud Functions

  • Pingback: Maritime noon report with Google Cloud Dataprep – Torbjorn Zetterlund ()

  • This guide has been a lifesaver! I opted out for using CircleCI to deploy because it seemed simpler than Cloud Build, but this made things a lot simpler to understand, especially for multiple cloud functions. Thanks!

  • This was an excellent post! Great idea, which I re-used – but implemented with Terraform instead of YAML files. Thanks again!

  • This should be on every front page on how to deploy from GitHub, yet everyone including Google is still telling developers to mirror the GitHub repo to a Cloud Source Repository.

    As a follow-up question, do you have any insights on how does this method actually works?

    Specifically, I thought it was mandatory to provide a location for the sources (think –source parameter). Since it is not present, I’m assuming it’s cloning the repo in whatever container Cloud Build uses and “Deploying from Your Local Machine” (https://cloud.google.com/functions/docs/deploying/filesystem). Any idea where this behavior (cloning the repo from GitHub) might be documented?

    Again, thank you so much for this. It definitely deserves WAY more exposure (took forever to find)

    • Torbjorn Zetterlund

      Hi Joan,

      I’m not aware of how the inner workings are on Google Cloud Platform, in many of my projects. I develop on the local machine (my laptop) I test the software I programmed on my local machine before I deploy. I use to deploy from my local machine Cloud Functions by using the gcloud command, I did not like that so much – I now deploy with Google Cloud build.

      Any cloud function code I push to GitHub, in Cloud Build I have set up a trigger to detect changes to the repository on GitHub if there is a new version the code is pulled from GitHub and deployed to the Cloud Function through the Cloud Build service.

      Please promote the article to your networks, so maybe one day it comes up first on a search.

      All the best
      Torbjorn

Leave a Reply to Mike Cancel Reply

Your email address will not be published.