Keeping a record of when was a table last modified is very crucial and important for us. Unlike MySql, Postgres doesn’t come with inbuilt functionality for this. So to solve this problem we will be using Triggers and Procedures. Triggers allow us to define functions that execute whenever a certain type…

Interestingly, I tried to backup database from RDS postgreSQL using pg_dump. However I cannot restore the data due to missing extensions (under routines table). For example I know these can be generated by single SQL. CREATE EXTENSION IF NOT EXISTS "uuid-ossp"; If I have custom-made extensions, I still need to…

There is more to Postgres performance than setting up indexes. — So you have set up your indexes correctly, but your API latency is still high due to the database. What else can you do? Well, you can reduce the number of database calls that you make. Read on. Queries have overhead Many things are going on when you query a database, such as…

It’s not the best tool for the job, but it might be the best for your situation — A message queue is a foundational architectural building block. Typically, it’s used for offloading workloads with high latency and needs resilience. For example, sending a receipt via email when a customer has placed an order. Many queue systems are available: RabbitMQ, Kafka, or Amazon SQS. Using a relational database as…

Exploring OpenAI Embedding API and Semantic search What we are gonna explore? Idea is to understand what is Semantic Search and Embedding is all about and try out a sample project using Node.Js to see both in action using OpenAI API. “Semantic search” is a search technique that aims to improve the accuracy and relevance…

Designing full-text search based on Postgres with event-driven architecture — Software projects of all sizes and complexity share the challenge of building scalable search solutions. Who hasn’t seen an application that runs searches using RDBMS SQL statements? You might be wondering if this is a good solution. Well, it depends. Performing searches using SQL could be sufficient for your use…

A Step-by-Step Guide to Discover and Harness the Power of Vector Databases — Table of Contents Intro What is so special about Vector Databases? How do we map the meaning of a sentence to a numerical representation? How does that help our LLM app? Why can’t we just give the LLM all the data we have? Hands-On Tutorial — Text to Embeddings and Distance Metrics 1…

Introduction: Full-Text Search searches the documents that match with a query and, optionally, sort the records according to their relevance to the query. A document, such as a magazine article, email, or database table serves as the search unit in a full-text search system. The most common type of search returns…

At Qonto, we noticed that it is powerful for business owners to grant cards for one-off expenses to their employees. That’s why we created Flash cards. With this feature, a customer can create a temporary card with a specific budget and time validity that corresponds to a specific need. …

Learn about Common Table Expression (CTE) and Window Functions — SQL is the bread and butter of every data professional. It doesn’t matter if you are a data analyst, data scientist, or data engineer, you need to have a solid understanding of how to write clean and efficient SQL queries. This is because behind any rigorous data analysis or any…

In this article, we’ll explore a powerful causal inference technique that I believe every data scientist should have in their toolbox. This article has six sections: 1) Introduction 2) Quick Causal Inference Primer 3) Brief Description of Causal Impact Algorithm 4) Applications of Counterfactual Inference with Time Series Data 5)…

Time Series Forecasting with LSTMs and GRUs: Credit Card Transaction Data — Time series analysis is the process of analyzing data points that are indexed by a timestamp or date. It is important for uncovering time dependencies such as upward, downward, seasonal, and cyclical trends. …