Articles by Shiv Iyer

About Shiv Iyer

Open Source Database Systems Engineer with a deep understanding of Optimizer Internals, Performance Engineering, Scalability and Data SRE. Shiv currently is the Founder, Investor, Board Member and CEO of multiple Database Systems Infrastructure Operations companies in the Transaction Processing Computing and ColumnStores ecosystem. He is also a frequent speaker in open source software conferences globally.

Redis Optimization: How to Tune Redis for High-Traffic Applications

Redis Optimization Guide for High-Traffic Applications Tuning Redis for high-traffic applications requires careful optimization of configuration, hardware, data structures, and monitoring. Here's a comprehensive guide to ensure Redis handles high traffic efficiently: 1. Redis Optimization [...]

Preventing PostgreSQL Transaction ID Wraparound

Preventing PostgreSQL Transaction ID Wraparound: Managing Autovacuum Blockages and Ensuring Database Health PostgreSQL Transaction ID Wraparound Transaction ID (XID) wraparound is a critical concern in PostgreSQL databases, as it can render a database unusable by [...]

Milvus Architecture and Internals

Milvus Architecture Milvus is a purpose-built vector database optimized for similarity search and AI workloads. Its architecture is distinct from traditional RDBMS like PostgreSQL and MySQL, as it is designed to handle high-dimensional vector data [...]

MongoDB Data Encryption: A Guide to Secure Data Management

Secure Data Management with MongoDB Data Encryption MongoDB's comprehensive data encryption framework encompasses three critical security layers: data at rest encryption, secure communication channels, and advanced key management protocols. This guide outlines the essential implementation [...]

Write Operation in Milvus: Scalable Optimization Strategies

Scalable Write Optimization Strategies for Milvus: Best Practices and Techniques Optimizing write operations in Milvus, an advanced vector database system, is essential for maintaining high performance in applications that require real-time data ingestion, large-scale data [...]

PostgreSQL Lost Updates: Understanding and Preventing Them

Understanding and Preventing PostgreSQL Lost Updates Executive Summary PostgreSQL Lost updates present a significant challenge in database management, particularly when handling concurrent transactions in PostgreSQL. This comprehensive guide examines the technical aspects of lost updates [...]

PostgreSQL Slow Queries: A PL/pgSQL Application to Log Them

Creating a PL/pgSQL Application to Log Slow Queries in PostgreSQL Server Creating a PostgreSQL application using PL/pgSQL to record all queries executed on the PostgreSQL server with a latency/response time of more than 1 second [...]

Using pgBadger for Continuous PostgreSQL Monitoring and Performance Optimization

Comprehensive Guide to Using pgBadger for Continuous PostgreSQL Monitoring and Performance Optimization Implementation of pgBadger for continuous PostgreSQL monitoring involves a systematic approach encompassing log configuration, analysis, and report generation. This advanced tool conducts a [...]

PostgreSQL WAL Files vs Transaction Logs: The Difference Explained

Understanding the Difference Between WAL Files and Transaction Logs in PostgreSQL In PostgreSQL, two key components that are frequently discussed in relation to database management and performance are the transaction log and the WAL (Write-Ahead [...]

PostgreSQL Ledger: Implementing a Tamper-Proof Ledger

Implementing a Tamper-Proof Ledger in PostgreSQL for Auditable and Secure Data Storage In PostgreSQL, a ledger refers to a specialized database structure or extension designed to provide immutable, auditable, and secure data storage. This structure [...]

The Data Transformation Company

Data Architecture, Engineering and Operations for SQL, NoSQL, NewSQL, Cloud Native Data Platforms, Analytics and AI