This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.
Tag: Big Data
-
What is Reverse ETL?
The data engineering industry has evolved from Extract, Transform, Load (ETL) to ELT, where raw data is copied from the source system loaded into a data warehouse or data lake and then transformed. The current trend is to adopt a new approach, called “reverse ETL,” the process of moving data from a data warehouse into third-party…
-
How to build a centralized and secured Data Catalog on Google Cloud Platform
Time is money, to organize your data in a Data Catalog is one of the first steps to give visibility to organizations’ data and to save time by employees to find data in the organization.
-
#datapipeline: Deploy Serverless Data pipeline to GCP with Terraform
I recently built a data pipeline which I wrote up in an article data pipeline the hourly electricity rate, what I did not explain was how I deployed it to the Google Cloud Platform. In this article, I will show you how I used Terraform to deploy it to the Google Cloud Platform. The biggest…
-
#Datapipeline: The T in ETL with SQL
If you read this I assume you are familiar with Extract Transform and Load (ETL), this is just a short article to show how you can do ETL and especially T, the transform using a SQL statement.
-
List of modern data stack technologies for your modern data stack
We have created a list of modern data stack technologies. The Modern Data Stack technologies are mostly SaaS-based technologies that comprise runs on a cloud-native data platform, generally leveraged to reduce the complexity in running a traditional data platform. The goal with the Modern Data Stack is to make data actionable by reducing the time…
-
Do you know how much CO2 your computer use?
Have you ever wondered how much Co2 your computer use and what impact computers have on climate change? I set out to investigate and came to the conclusion that it is now easy to figure, accordingly to Worldometer there has been shipped over 2 billion computers as of 2007, the number is high. This year…
-
What uses the most electricity in your home?
This year when the electricity prices skyrocket in my village, I wanted to find out how much electricity my appliances consumed, and while I was at it I also calculated the carbon impact of my electrical appliances.
-
Data Engineering Roadmap för 2021
Kanske är det sexsiffriga löner, möjligheten att arbeta med cool teknik eller att folk äntligen lär sig att datateknik är där allt börjar inom dataområdet. Oavsett orsaken märker folk. VC:er investerar i datalagrings- och intagsplattformar och företag intervjuar fler dataingenjörer jämfört med tidigare år.
-
Data Project: Python with Google Analytics API
In this article, I will take you through the details of writing a Python Notebook code to extract data from Google Analytics (GA4) using the Google API Python library to access Google Analytics.