ETL Strategies & Pipelines have now become inevitable for cloud business needs. There are several ETL tools in the market ranging from open-source ones such as Airflow, Luigi, Azkaban, Oozie to enterprise solutions such as Azure Data Factory, AWS Glue, Alteryx, Fivetran, etc. But what makes the data pipelines to…

This universe is governed by a cosmic duality, sets of two opposing and complementing principles, or cosmic energies that can be observed in nature. There’s Yin and there’s Yang…

There’s relational and there are non-relational databases…

And of course, the visualization tools; There’s Tableau Server and there’s Power BI Embedded.!!

The Yin & Yang of the present BI universe

Postgres, one of the widely used Relational Database Management System; has been widely adopted due to its ability to handle different workloads such as web services, warehouses, etc.

Fun Fact: The name Postgres comes from it’s predecessor originated from UC Berkley’s Ingres Database (INteractive GRaphics iterchangE System; meaning it’s Post-INGRES).

& How not to accidentally Rollback all of your migrations

Flyway — by boxfuse: This is a schema migration tool and it acts more of like version control for your relational databases.

If you are manually executing your SQL scripts or if your administrator is manually executing the SQL scripts, on your production or UAT environment, you definitely need this…

++ Cons on ETL pipelines

Just another day at the work place;

5 minutes post the boot:

You hear everyone complain that the production database is slow. You quickly start to investigate; exploring all possible outcomes on the dashboards..

Could it have been the long-running slow query which you had raised a ticket for the…

The No-BS guide

Before we begin.. Here are a few basics.

Analyzer:

An analyzer does the analysis or splits the indexed phrase/word into tokens/terms upon which the search is performed with much ease.

An analyzer is made up of tokenizers and filters. …

The following are some of the instances where the developers simply love to hate!

  • The one-last-thing syndrome — This reminds me of the following quote:

The first 90 percent of the code accounts for the first 90 percent of the development time. …

The question is.. What brings you here? Fed up with all the searches on how to back-up and restore specific indices?

Fear not, for your search quest ends here.!

After going through a dozens of tiny gists and manual pages, here it is.. We’ve done all the heavy-lifting for you.

DATABASE MIGRATION ACROSS PLATFORMS — Got your goosebumps yet?

Well, long story short; cross-platform database migrations equals sleep talking, distress and long day works with coffee; and what good does it do? We will just end up writing hours and hours of scripts to conquer the end-result. …

Yet another post about MNIST Handwritten Digits Prediction?

Nope. Not this time!!

There are about a hundred of tutorials available on-line for this cause.
Here’s a quickie, to understand all the mechanics of the prediction process in tensorflow for the MNIST datasets which should get you, up and running.

Done…

Sathiya Sarathi

Data &Cloud Engineer; Author @ https://www.datawrangler.in

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store