Scaling Postgres

In this episode of Scaling Postgres, we discuss different pagination solutions, how to return modifications, soft deletion alternatives and how to use JSON with PostgreSQL. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/pagination-problem-total-result-count/ https://sqlfordevs.com/returning-modified-rows https://brandur.org/fragments/deleted-record-insert https://dev.to/ftisiot/how-to-json-in-postgresqlr-1lj7 https://momjian.us/main/writings/pgsql/beyond.pdf http://peter.eisentraut.org/blog/2023/01/11/postgresql-largest-commits https://supabase.com/blog/postgres-foreign-data-wrappers-rust https://medium.com/@walttonm/using-the-window-function-row-number-to-remove-duplicates-in-postgresql-5aef1edfb78c https://pgdash.io/blog/horizontally-scaling-postgresql.html https://www.percona.com/blog/upgrading-postgresql-extensions/ https://www.timescale.com/blog/a-postgresql-developers-perspective-six-interesting-patches-fr

In this episode of Scaling Postgres, we discuss how to generate test data, how WAL archiving is faster in PG15, how to set date statistics on a timestamp and how to avoid useless indexes. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://kmoppel.github.io/2022-12-23-generating-lots-of-test-data-with-postgres-fast-and-faster/ https://www.percona.com/blog/speed-up-of-the-wal-archiving-in-postgresql-15/ https://www.cybertec-postgresql.com/en/improving-group-by-with-create-statistics/ https://mydbanotebook.org/post/too-many-indexes/ https://www.cybertec-postgresql.com/en/json-logs-in-postgresql-15/ http://peter.eisentraut.org/blog/2022/12/27/postgresql-hidden-gems https://www.softwareandbooz.com/advent-of-code-2022-days-6-10-with-postgresql/ https://www.crunchydata.com/blog/solving-advent-of-code-2022-using-postgres-day-6 https://www.crunchydata.com/blog/solving-advent-of-code-2022-using-postgres-day-7 https://www.crunchydata.com/blog/solving-a

In this episode of Scaling Postgres, we discuss columnar table storage, solving Advent of Code using Postgres, new features coming in Postgres 16 and when Postgres development happens. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.percona.com/blog/powering-postgresql-15-with-columnar-tables/ https://github.com/citusdata/citus/blob/main/src/backend/columnar/README.md https://www.softwareandbooz.com/advent-of-code-2022-with-postgresql-part1/ https://www.crunchydata.com/blog/solving-advent-of-code-2022-using-postgres-day-1 https://www.crunchydata.com/blog/solving-advent-of-code-2022-using-postgres-day-2 https://www.crunchydata.com/blog/solving-advent-of-code-2022-using-postgres-day-3 https://www.crunchydata.com/blog/solving-advent-of-code-2022-using-postgres-day-4 https://www.crunchydata.com/blog/solving-advent-of-code-2022-using-postgres-day-5 https://www.depesz.com/2022/12/14/waiting-for-postgresql-16-add-grantable-maintain-privilege

In this episode of Scaling Postgres, we discuss ENUMs vs. check constraints, querying table facets with roaring bitmaps, a better way to handle scaling and whether you should store your business logic in Postgres. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.crunchydata.com/blog/enums-vs-check-constraints-in-postgres https://www.cybertec-postgresql.com/en/faceting-large-result-sets/ https://github.com/cybertec-postgresql/pgfaceting https://pganalyze.com/blog/5mins-postgres-roaring-bitmaps-pgfaceting-query-performance https://thenewstack.io/iso-better-scaling-instacart-drops-postgres-for-amazon-dynamodb/ https://www.rubberduckdevshow.com/episodes/68-should-you-store-business-logic-in-your-database/ https://www.ongres.com/blog/debugging-postgres-wal-events-with-pg_walinspect/ https://kmoppel.github.io/2022-12-09-the-bountiful-world-of-postgres-indexing-options/ https://andreas.scherbaum.la/blog/archives/1125-PGSQL-Phriday-003-What-is-

In this episode of Scaling Postgres, we discuss merging indexes vs. a composite index, implementing transparent column encryption, developing trusted language extensions, and reviewing the WAL archive module. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://sirupsen.com/index-merges https://supabase.com/blog/transparent-column-encryption-with-postgres https://aws.amazon.com/blogs/aws/new-trusted-language-extensions-for-postgresql-on-amazon-aurora-and-amazon-rds/ https://pganalyze.com/blog/5mins-postgres-pg-tle-custom-extensions-aws-rds-aurora-trusted-languages-hooks https://github.com/aws/pg_tle https://www.percona.com/blog/new-wal-archive-module-library-in-postgresql-15/ https://sqlasylum.wordpress.com/2022/11/29/pgsql-phriday-003-what-is-the-community-to-you/ https://www.softwareandbooz.com/postgresql-community-passionate-knowledgeable-and-thoughtful/ https://www.scarydba.com/2022/12/02/pgsql-phriday-3-what-is-the-postgresql-community-to-y

In this episode of Scaling Postgres, we discuss how PG15 helps reduce replication lag, how to get the parameters used in prepared statements, PostGIS Day and how to use multiple pgbouncers. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://pganalyze.com/blog/5mins-postgres-performance-in-lists-vs-any-operator-bind-parameters https://sqlfordevs.com/ghost-conditions-for-unindexed-columns https://www.crunchydata.com/blog/percentage-calculations-using-postgres-window-functions https://www.postgresonline.com/journal/index.php?/archives/406-VARIADIC-Unnest.html https://supabase.com/blog/sql-or-nosql-both-with-postgresql https://www.cybertec-postgresql.com/en/error-permission-denied-schema-public/ https://www.highgo.ca/2022/11/25/global-index-benchmark-with-pgbench/ https://fluca1978.github.io/2022/11/24/PostgreSQLDataSyncRetry.html https://postgres.fm/episodes/materialized-views https://postgresql.life/post/tobias_bussmann/  

In this episode of Scaling Postgres, we discuss how PG15 helps reduce replication lag, how to get the parameters used in prepared statements, PostGIS Day and how to use multiple pgbouncers. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://pganalyze.com/blog/5mins-postgres-15-maintenance-io-concurrency-reduce-replication-lag https://www.cybertec-postgresql.com/en/explain-that-parameterized-statement/ https://blog.rustprooflabs.com/2022/11/route-the-interesting-things-postgis-day2022 https://www.crunchydata.com/blog/postgres-at-scale-running-multiple-pgbouncers https://www.crunchydata.com/blog/postgres-query-boost-using-any-instead-of-in https://fluca1978.github.io/2022/11/16/psqlEmacsClient.html https://people.planetpostgresql.org/devrim/index.php?/archives/121-Improved-aarch64-arm64-support-is-available-in-the-PostgreSQL-RPM-repository..html https://postgrespro.com/blog/pgsql/5969859 https://www.crunchydata.com/blog/multi-cloud-strategies-wi

In this episode of Scaling Postgres, we discuss the release of Postgres 15.1, a Postgres 10 retrospective, how to reduce replication lag and using listen & notify. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-151-146-139-1213-1118-and-1023-released-2543/ https://jkatz05.com/post/postgres/postgres-10-tribute/ https://kmoppel.github.io/2022-11-09-postgres-v15-a-billion-transactions-later/ https://www.citusdata.com/blog/2022/11/10/reducing-replication-lag-with-io-concurrency-in-pg15/ https://www.enterprisedb.com/blog/listening-postgres-how-listen-and-notify-syntax-promote-high-availability-application-layer http://amitkapila16.blogspot.com/2022/11/logical-replication-improvements-in.html https://pganalyze.com/blog/5mins-postgres-15-logical-replication-replica-identity https://www.postgresql.fastware.com/blog/performance-impact-of-row-filters-and-column-lists-in-logical-replication https://www.ti

In this episode of Scaling Postgres, we discuss how sequential scans can be forced, the best way to rename a table without downtime, different Postgres terminology and the PGSQL Phriday blogging event. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://code.jeremyevans.net/2022-11-01-forcing-sequential-scans-on-postgresql.html https://pganalyze.com/blog/5mins-postgres-large-integers-causing-sequential-scan-instead-of-using-index https://brandur.org/fragments/postgres-table-rename https://www.crunchydata.com/blog/challenging-postgres-terminology https://andreas.scherbaum.la/blog/archives/1122-PGSQL-Phriday-002-PostgreSQL-Backup-and-Restore.html https://blog.rustprooflabs.com/2022/11/pgsql-phriday-002 https://mydbanotebook.org/post/how-to-do-proper-backups/ https://www.scarydba.com/2022/11/04/validating-backups-pgsql-phriday-002/ https://www.softwareandbooz.com/adventures-in-postgresql-backups/ http://rhaas.blogspot.com/2022/11/pgbasebackup-coul

In this episode of Scaling Postgres, we discuss how to set shared buffers, the different WAL compression algorithm options, how to use the merge command and how to contribute to Postgres. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.enterprisedb.com/blog/harnessing-shared-buffers-part-2 https://pganalyze.com/blog/5mins-postgres-tuning-shared-buffers-pgbench-TPROC-C-TPROC-H https://www.enterprisedb.com/blog/you-can-now-pick-your-favorite-compression-algorithm-your-wals https://www.enterprisedb.com/blog/merge-command https://postgres.fm/episodes/contributing-to-postgres https://supabase.com/blog/postgresql-commitfest http://peter.eisentraut.org/blog/2022/10/25/postgresql-15-statistics https://www.highgo.ca/2022/10/28/cross-partition-uniqueness-guarantee-with-global-unique-index/ https://www.crunchydata.com/blog/postgres-databases-and-schemas https://www.crunchydata.com/blog/easier-upgrades-and-image-management-for-postgres-in-kubernetes

In this episode of Scaling Postgres, we discuss why and what to do when updates are slow, use cases for lateral joins, moving from serial to identity for auto-incrementing ids and changes to Postgres 15. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/why-are-my-postgresql-updates-getting-slower/ https://pganalyze.com/blog/5mins-postgres-debug-UPDATE-bloated-tables-auto-explain-pageinspect https://sqlfordevs.com/for-each-loop-lateral-join https://adamj.eu/tech/2022/10/21/migrate-postgresql-ids-serial-identity-django-4.1/ https://blog.rustprooflabs.com/2022/10/postgres-15-config-changes https://pgconfig.rustprooflabs.com/param/change/14/15 https://www.enterprisedb.com/blog/exclusive-backup-mode-finally-removed-postgres-15 https://fluca1978.github.io/2022/10/21/PostgreSQL15JsonLogs.html https://www.enterprisedb.com/blog/postgres-15-adds-copy-header-and-matching https://www.endpointdev.com/blog/2022/10/upgrading-

In this episode of Scaling Postgres, we discuss the release of Postgres 15, the performance of different file systems with Postgres, options for connection pooling and how to secure your connections. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-15-released-2526/ https://www.crunchydata.com/blog/be-ready-public-schema-changes-in-postgres-15 https://www.crunchydata.com/blog/a-look-at-postgres-15-merge-command-with-examples https://www.enterprisedb.com/blog/postgres-vs-file-systems-performance-comparison https://www.cybertec-postgresql.com/en/pgbouncer-types-of-postgresql-connection-pooling/ https://tailscale.com/blog/introducing-pgproxy/ https://www.timescale.com/blog/vpc-peering-from-zero-to-hero/ https://www.softwareandbooz.com/pgsql-phriday-001-wrap-up-and-review/ https://knock.app/blog/postgres-index-findings https://pganalyze.com/blog/5mins-postgres-15-managing-indexes https://www.percona.com/b

In this episode of Scaling Postgres, we discuss the Postgres 15 RC2, optimizing shared buffers, how to secure your database connections and blog posts from PGSQL Phriday. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-15-rc-2-released-2521/ https://www.enterprisedb.com/blog/harnessing-shared-buffers-and-reaping-performance-benefits-part-1 https://innerjoin.bit.io/the-majority-of-postgresql-servers-on-the-internet-are-insecure-f1e5ea4b3da3 https://www.softwareandbooz.com/pgsql-phriday-001-invite/ https://andreas.scherbaum.la/blog/archives/1121-PGSQL-Phriday-001-Two-truths-and-a-lie.html https://mydbanotebook.org/post/2-it-depends-and-1-must/ https://andyatkinson.com/blog/2022/10/07/pgsqlphriday-2-truths-lie https://www.scarydba.com/2022/10/07/pgsql-phriday-001-two-truths-and-a-lie/ https://sqlasylum.wordpress.com/2022/10/07/pgsqlphriday-001-truth-and-lies/ https://hakibenita.com/future-proof-sql htt

In this episode of Scaling Postgres, we discuss the release of the Postgres 15 Release Candidate 1, new ICU collations features, how to use listen & notify and understanding TOAST. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-15-rc-1-released-2516/ http://peter.eisentraut.org/blog/2022/09/26/icu-features-in-postgresql-15 https://pganalyze.com/blog/5mins-postgres-15-logical-replication-filters-database-wide-icu-collations https://www.cybertec-postgresql.com/en/listen-notify-automatic-client-notification-in-postgresql/ https://www.crunchydata.com/blog/real-time-database-events-with-pg_eventserv https://www.dbi-services.com/blog/toasting-strategies-in-postgresql/ https://www.dbi-services.com/blog/toasting-in-postgresql-toast-tables/ https://www.dbi-services.com/blog/toasting-in-postgresql-lets-see-it-in-action/ https://blog.rustprooflabs.com/2022/10/announce-mastering-postgis-openstreemap https

In this episode of Scaling Postgres, we discuss using rust for Postgres extensions, performance comparisons of TimescaleDB vs. Postgres, uninterrupted writes when sharding in Citus and the Postgres data flow. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://postgresml.org/blog/postgresml-is-moving-to-rust-for-our-2.0-release/ https://www.timescale.com/blog/postgresql-timescaledb-1000x-faster-queries-90-data-compression-and-much-more/ https://www.citusdata.com/blog/2022/09/19/citus-11-1-shards-postgres-tables-without-interruption/ https://www.crunchydata.com/blog/postgres-data-flow https://www.cybertec-postgresql.com/en/postgresql-sequences-vs-invoice-numbers/ https://postgrespro.com/blog/pgsql/5969741 https://www.crunchydata.com/blog/fun-with-postgres-functions https://www.depesz.com/2022/09/18/what-is-lateral-what-is-it-for-and-how-can-one-use-it/ https://www.postgresql.fastware.com/blog/column-lists-in-logical-replication-publications https

In this episode of Scaling Postgres, we discuss little known useful features, a monthly community blogging initiative, PG 15 Anticipation, and getting generic plans. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://it.badykov.com/blog/2022/09/12/simple-and-usefull-postgresql-features/ https://www.softwareandbooz.com/introducing-psql-phriday/ https://www.youtube.com/watch?v=b-C3TjmzEmU https://dev.to/yugabyte/explain-from-pgstatstatements-normalized-queries-how-to-always-get-the-generic-plan-in--5cfi https://pganalyze.com/blog/5mins-postgres-explain-pg-stat-statements-plan-cache-mode-normalized-query https://www.cybertec-postgresql.com/en/what-is-an-inner-join-in-sql-and-what-is-an-outer-join/ https://proopensource.it/blog/json-versus-jsonb https://www.percona.com/blog/working-with-snapshots-in-postgresql/ https://www.crunchydata.com/blog/postgres-pgcrypto https://www.citusdata.com/blog/2022/09/12/distributed-postgres-goes-full-open-source-wit

In this episode of Scaling Postgres, we discuss the release of Postgres 15 Beta 4, different primary key options, sequence limits and how to examine your Postgres configuration. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-15-beta-4-released-2507/ https://supabase.com/blog/choosing-a-postgres-primary-key#the-post-uuidv1v4-era-a-cambrian-explosion-of-identifiers https://blog.devgenius.io/analyzing-new-unique-identifier-formats-uuidv6-uuidv7-and-uuidv8-d6cc5cd7391a https://www.2ndquadrant.com/en/blog/sequential-uuid-generators/ https://www.cybertec-postgresql.com/en/error-nextval-reached-maximum-value-of-sequence/ https://proopensource.it/blog/postgresql-configuration https://aws.amazon.com/blogs/database/parallel-vacuuming-in-amazon-rds-for-postgresql-and-amazon-aurora-postgresql/ https://pganalyze.com/blog/5mins-postgres-parallel-vacuum-sql-json-postgres-16 https://blog.jooq.org/how-to-plot-an-asci

In this episode of Scaling Postgres, we discuss the PG15 changes to public schema permissions, experiences with kubernetes for Postgres management, using dump manifests for restores and a fork of PgBouncer. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://andreas.scherbaum.la/blog/archives/1120-Changes-to-the-public-schema-in-PostgreSQL-15-and-how-to-handle-upgrades.html https://proopensource.it/blog/postgresql-on-k8s-experiences https://pganalyze.com/blog/5mins-postgres-kubernetes-operator-handling-major-version-upgrades https://www.percona.com/blog/working-with-postgresql-dump-manifests/ https://blog.cloudflare.com/open-sourcing-our-fork-of-pgbouncer/ https://www.postgresql.fastware.com/blog/how-postgresql-15-improved-communication-in-logical-replication https://www.depesz.com/2022/09/02/sql-json-is-postponed/ https://www.cybertec-postgresql.com/en/postgresql-alter-table-add-column-done-right/ https://www.crunchydata.com/blog/postgres-migra

In this episode of Scaling Postgres, we discuss how the stats collector disappears in PG15, steps to mitigate high latency connections, how to run Postgres in the browser and the future of high availability. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.percona.com/blog/postgresql-15-stats-collector-gone-whats-new/ https://pganalyze.com/blog/5mins-postgres-improve-query-network-latency-performance-pipeline-mode-copy-tc https://www.crunchydata.com/blog/crazy-idea-to-postgres-in-the-web-browser https://www.enterprisedb.com/blog/pg-phriday-defining-high-availability-postgres-world https://www.enterprisedb.com/blog/ansible-benchmark-framework-postgresql https://www.timescale.com/blog/what-does-a-postgresql-commitfest-manager-do-and-should-you-become-one/ https://postgis.net/2022/08/25/tip-upgrading-postgis-sfcgal/ https://www.cybertec-postgresql.com/en/migrate-scheduled-jobs-to-pg_timetable-from-pgagent/ https://postgres.fm/episodes/how-t

In this episode of Scaling Postgres, we discuss a new Postgres playground, a book about PG14 internals, how to corrupt your database and using anti-joins. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.crunchydata.com/blog/learn-postgres-at-the-playground https://www.crunchydata.com/developers/tutorials https://www.youtube.com/watch?v=Ryj5c8zLF50 https://postgrespro.com/blog/pgsql/5969682 https://www.cybertec-postgresql.com/en/how-to-corrupt-your-postgresql-database/ https://www.crunchydata.com/blog/rise-of-the-anti-join https://www.enterprisedb.com/blog/aws-rds-postgresql-deployment-pgadmin-4 https://www.enterprisedb.com/blog/leverage-new-way-import-existing-postgres-database-kubernetes https://postgrespro.com/blog/pgsql/5969676 https://www.cybertec-postgresql.com/en/aliases-for-sub-selects-in-from-clause/ https://postgres.fm/episodes/monitoring-checklist https://pganalyze.com/blog/5mins-postgres-security-patch-releases-pgspot-pghostil