This was a very interesting project that I worked on, the use case for the project was to make the maritime noon report data available in Google BigQuery, sounds simple enough. However there were a lot of requirements, from collecting application data from the Greenpeace fleet in different formats leveraging Cloud Functions and API calls, landing the data in Cloud Storage, normalizing the data for reporting and translating each different ships’ schemas into one common schema in BigQuery thanks to …
Tag: Converting data
I wanted to find out how polluted the air is around me, and if it had improved now when we work from home and there are fewer airplanes going pass or landing or departing from Schiphol …
Google has added the SQL server as an option to the Google Cloud SQL service. In this article, I will explore how to set up a Google Cloud SQL server I received a question what was …
This is to follow up on my previous article Detecting non newline delimited JSON file with Google Cloud Dataprep in this article I will show you how to create a Cloud Dataprep recipe that transforms the …
I was recently working on a data project that required us to request data from an internal application using a RESTful API and transform the data to be stored in the Google data warehouse solution BigQuery. …
BigQuery is the Google Cloud Platform data warehouse solution, and setting up datasets you have many options for importing data into a BigQuery table. There are times when you just want to empty a table, to …
Scrolling my news feed this morning, there was one thing that caught my eye, the news I read was that Microsoft had open-source their visualization data explorer tool SandDance. …
If you are following what is going on in BigData, you know that there is current chaos in the BigData ecosystem. …
I have been doing a lot of work with creating Jupyter notebooks and collaborating with the Google Colab service, Colaboratory or short-form Colab is a free Jupyter notebook environment that requires no setup and runs entirely …
Have you ever wanted to visualize any of your data showing a bar graph, pie chart, word clouds, and other web visualization more than the text? There been an explosion in the number of available data sources …
There is no straight way of converting data from MySQL to MongoDB, but there are a few techniques that you can use. …
