Tag: Big Data

  • What is Reverse ETL?

    What is Reverse ETL?

    The data engineering industry has evolved from Extract, Transform, Load (ETL) to ELT, where raw data is copied from the source system loaded into a data warehouse or data lake and then transformed. The current trend is to adopt a new approach, called “reverse ETL,” the process of moving data from a data warehouse into third-party…

  • How to build a centralized and secured Data Catalog on Google Cloud Platform

    How to build a centralized and secured Data Catalog on Google Cloud Platform

    Time is money, to organize your data in a Data Catalog is one of the first steps to give visibility to organizations’ data and to save time by employees to find data in the organization.

  • #datapipeline: Deploy Serverless Data pipeline to GCP with Terraform

    #datapipeline: Deploy Serverless Data pipeline to GCP with Terraform

    I recently built a data pipeline which I wrote up in an article data pipeline the hourly electricity rate, what I did not explain was how I deployed it to the Google Cloud Platform. In this article, I will show you how I used Terraform to deploy it to the Google Cloud Platform. The biggest…

  • #Datapipeline: The T in ETL with SQL

    #Datapipeline: The T in ETL with SQL

    If you read this I assume you are familiar with Extract Transform and Load (ETL), this is just a short article to show how you can do ETL and especially T, the transform using a SQL statement.

  • List of modern data stack technologies for your modern data stack

    List of modern data stack technologies for your modern data stack

    We have created a list of modern data stack technologies. The Modern Data Stack technologies are mostly SaaS-based technologies that comprise runs on a cloud-native data platform, generally leveraged to reduce the complexity in running a traditional data platform. The goal with the Modern Data Stack is to make data actionable by reducing the time…

  • Do you know how much CO2 your computer use?

    Do you know how much CO2 your computer use?

    Have you ever wondered how much Co2 your computer use and what impact computers have on climate change? I set out to investigate and came to the conclusion that it is now easy to figure, accordingly to Worldometer there has been shipped over 2 billion computers as of 2007, the number is high. This year…

  • What uses the most electricity in your home?

    What uses the most electricity in your home?

    This year when the electricity prices skyrocket in my village, I wanted to find out how much electricity my appliances consumed, and while I was at it I also calculated the carbon impact of my electrical appliances.

  • 23360 days old and counting

    23360 days old and counting

    I achieved 64 years (23360 days) this week. Here are some brilliant reflections: what is life about? I think a balance between knowledge building and connections. I did not know what I wanted to do in my life before I was introduced to software development. Before that, I worked in construction, as a delivery truck…

  • Data Engineering Roadmap för 2021

    Data Engineering Roadmap för 2021

    Kanske är det sexsiffriga löner, möjligheten att arbeta med cool teknik eller att folk äntligen lär sig att datateknik är där allt börjar inom dataområdet. Oavsett orsaken märker folk. VC:er investerar i datalagrings- och intagsplattformar och företag intervjuar fler dataingenjörer jämfört med tidigare år.

  • Data Project: Python with Google Analytics API

    Data Project: Python with Google Analytics API

    In this article, I will take you through the details of writing a Python Notebook code to extract data from Google Analytics (GA4) using the Google API Python library to access Google Analytics.