Norway ice fishers Tromso

Using Cloud Build as your CI/CO for multiple Cloud Functions

There are many CI/CO tools available in the market today like GitLab CI, CircleCI I’m not so much in favor of these CI/CO services even if they have some cool features. I have been a Google Cloud Platform user for a few years now, and I prefer to use the tools that are available to me by a cloud platform that I use.

In my case, for CI/CO I’m using Google Cloud Build that is available in the Google Cloud Platform, in this how-to I will explain how I use Google Cloud Build to automatically deploy cloud functions to Google Cloud Function when an update is pushed to a GitHub repo.

Cloud Functions

I created a service to identify and track brand name being used and sold by illegal merchants on online platforms, for this I had put together a cloud architecture on the Google Cloud Platform that used serverless computing. I created a number of cloud functions Bots that run on the Google Cloud Function platform. The architecture looked like this.

As I had more than one cloud function, I wanted to keep all the cloud functions in one GitHub repository, and only the cloud function that was changed would be deployed by a trigger in the Cloud Build.

Organize cloud functions on GitHub

The way I organized the GitHub repo was by creating sub-folders for each of my cloud functions. It looked like this:

github sub-directories
github sub-directories

In each of the sub-directories, I created a cloudbuild.yaml file that has the instructions for Cloud Build on how to build and deploy. The cloudbuild.yaml looks like this.

steps:
  - name: "gcr.io/cloud-builders/gcloud"
    waitFor: ["-"]
    args:
      - functions
      - deploy
      - googlecloudbot_http
      - --runtime=python37
      - --trigger-http
      - --region=europe-west1
      - --memory=128MB
    id: "deploying-a-serverless-function"
    dir: "gpi-techlab-googlesearchbot"

Google Cloud Build

In the Google Cloud Platform Console, I selected Cloud Build. What I needed to do is to create a trigger that starts the build action to deploy a new version. I choose to Push to a branch as the Event for the main branch, in my case Master. This would be different depending on the size of the team working with the branch, in my case, it’s just me. I would suggest to Push a new tag if a team is working on the GitHub branch.

Google cloud build trigger
Google cloud build trigger

I created a trigger for each functions sub-directories in the GitHub branch, to only make that sub-directory to deploy I used cloudbuild.yaml inside each GitHub branch sub-directory. If the trigger was to detect a code change on the GitHub branch, I wanted it only to be trigger on the sub-directory that the code change to start the build process.

I achieved that by creating what include files filter I wanted, I specified the include file filter with the sub-directory – as shown in the example below I specified gpi-techlab-bingboot/** that would only include all files from the sub-directory gpi-techlab-bingboot

And I also specified that I had a .yaml file and that the .yaml file could be found in the sub-directory gpi-techlab-bingboot

Conclusion

That is all for this how-to, it is straight forward, and it works excellent. If you have questions you can contact me or you can read more about Google Cloud Build triggers here.


by

Comments

13 responses to “Using Cloud Build as your CI/CO for multiple Cloud Functions”

  1. […] The Cloud Function GitHub REPO comes with an .yaml file so you can setup an automatic deployment build process – you can read more how to do in this article Using Cloud Build as you CI/CO for multiple Cloud Functions […]

  2. Nison M. Avatar
    Nison M.

    This guide has been a lifesaver! I opted out for using CircleCI to deploy because it seemed simpler than Cloud Build, but this made things a lot simpler to understand, especially for multiple cloud functions. Thanks!

    1. Torbjorn Zetterlund Avatar
      Torbjorn Zetterlund

      Thanks for the kind words about how it helped you out.

  3. Mike Avatar
    Mike

    This was an excellent post! Great idea, which I re-used – but implemented with Terraform instead of YAML files. Thanks again!

  4. Joan Avatar
    Joan

    This should be on every front page on how to deploy from GitHub, yet everyone including Google is still telling developers to mirror the GitHub repo to a Cloud Source Repository.

    As a follow-up question, do you have any insights on how does this method actually works?

    Specifically, I thought it was mandatory to provide a location for the sources (think –source parameter). Since it is not present, I’m assuming it’s cloning the repo in whatever container Cloud Build uses and “Deploying from Your Local Machine” (https://cloud.google.com/functions/docs/deploying/filesystem). Any idea where this behavior (cloning the repo from GitHub) might be documented?

    Again, thank you so much for this. It definitely deserves WAY more exposure (took forever to find)

    1. Torbjorn Zetterlund Avatar
      Torbjorn Zetterlund

      Hi Joan,

      I’m not aware of how the inner workings are on Google Cloud Platform, in many of my projects. I develop on the local machine (my laptop) I test the software I programmed on my local machine before I deploy. I use to deploy from my local machine Cloud Functions by using the gcloud command, I did not like that so much – I now deploy with Google Cloud build.

      Any cloud function code I push to GitHub, in Cloud Build I have set up a trigger to detect changes to the repository on GitHub if there is a new version the code is pulled from GitHub and deployed to the Cloud Function through the Cloud Build service.

      Please promote the article to your networks, so maybe one day it comes up first on a search.

      All the best
      Torbjorn

  5. Mithilesh K Bhagat Avatar
    Mithilesh K Bhagat

    This was an great post! Nicely explained.

    The only drawback i see with this approach is creating a trigger for each functions.
    Is there a way we can ignore creating multiple CF and have only signle triger which listens/monitors to the changes in sub directories and deploy only the updated or newly added CF.

  6. Mithilesh K Bhagat Avatar
    Mithilesh K Bhagat

    This was an excellent post! Nicely explained.

    The only drawback i see with this approach is creating a trigger for each functions.
    Is there a way we can ignore creating multiple triggers and have only signle trigger which listens/monitors to the changes in sub directories and deploy only the updated or newly added CF.

    1. momshomecooking Avatar
      momshomecooking

      You could take a look at using Terraform to deploy with, I have moved my functions to Terraform still and still use GitHub as a code repository.

  7. Ackeley Avatar
    Ackeley

    Hi Torbjörn Zetterlund!

    excellent tutorial, I have a case that I need when I run the build, it makes a git diff and loads only the changes and not all the repositories, do you know if this is possible, without using terraform?

    1. torbjorn Avatar

      Not sure, have not had that use case before and just now I do not have the time to look into it.

  8. […] this blog post by Torbjorn Zetterlund you can read the entire process of deploying multiple cloud […]

  9. […] have setup a cloud build trigger using the directions here. I want have it setup so that any merge onto the main branch of my GitHub repo will start a Cloud […]

Leave a Reply to Torbjorn Zetterlund Cancel reply

Your email address will not be published. Required fields are marked *