Writings.
-
Extracting data from Bigquery into Redsfhit
Another morning, another challenge! :) This time I have to integrate data collected into Google Ads into our data warehouse based on Redshift. I started with Google and find several tricky ways to get data into Redshift. Some paid ETL services. Self-hosted script to capture data from Google Ads directly via…
-
How do we build analytics system in Scentbird
Couple of months ago Scentbird has celebrated 2 years of building internal analytics system. I considered it is a time to start sharing our experience with the community and it was a topic of my HighLoad 2019 (the biggest conference in Russia regarding high load systems) speech. Here it is…
-
Jenkins: How to start Jenkins build using commit id of last successful build on test enviroment?
Last week I faced with a quite not trivial case in CI practice. There is production build running once a day by schedule and it is using master branch as a source. The main challenge here that the development team is not using branches in everyday practice (why? it is…
-
Generating series in Redshift
Redshift in 99% of cases means data warehouse, where generating series (especially date series) is quite a common case. You will be disappointed to find out no information about series in official documentation (the only thing that you will be able to find - "generate_series function is not…
-
Running static Iglu repository on AWS S3
During setuping Snowplow analytics systems I have to setup a private Iglu repository. The main idea behind this is described on (https://github.com/snowplow/iglu/wiki/Setting-up-an-Iglu-repository). That manual missed several steps that are really important for building Iglu repository on AWS infrastructure. I had spent a lot of…