Scaling Postgres

In this episode of Scaling Postgres, we discuss new Postgres releases, a new privilege escalation CVE, chaos testing a high availability kubernetes cluster as well as addressing other H/A questions. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-145-138-1212-1117-1022-and-15-beta-3-released-2496/ https://www.enterprisedb.com/blog/postgresql-extensions-impacted-cve-2022-2625-privilege-escalation https://coroot.com/blog/chaos-testing-zalando-postgres-operator https://www.timescale.com/blog/how-high-availability-works-in-our-cloud-database/ https://www.enterprisedb.com/blog/pg-phriday-dos-and-donts-postgres-high-availability-qa https://pganalyze.com/blog/5mins-postgres-linux-readahead-effective-io-concurrency https://smallthingssql.com/having-a-less-understood-sql-clause https://postgrespro.com/blog/pgsql/5969673 https://postgres.fm/episodes/vacuum https://postgresql.life/post/adam_wright/ https://www.r

In this episode of Scaling Postgres, we discuss PG14's new SQL function In this episode of Scaling Postgres, we discuss research into a performance puzzle, results from a Postgres and pgAdmin survey as well as the impacts of long running queries on Aurora and Postgres. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://ardentperf.com/2022/07/31/researching-the-performance-puzzle/ https://www.timescale.com/state-of-postgres/2022 https://www.enterprisedb.com/blog/pgadmin-user-survey-2022 https://aws.amazon.com/blogs/database/manage-long-running-read-queries-on-amazon-aurora-postgresql-compatible-edition/ https://pganalyze.com/blog/5mins-postgres-lock-conflicts-amazon-aurora-max-standby-streaming-delay-hot-standby-feedback https://www.highgo.ca/2022/08/05/how-to-set-up-nfs-and-run-pg-on-it/ https://postgresql.life/post/beena_emerson/ https://www.rubberduckdevshow.com/episodes/53-discussing-no-more-estimates-jason-charnes/  

In this episode of Scaling Postgres, we discuss PG14's new SQL function syntax, including buffers when doing explain analyze, how to debug autovacuum and using row-level security to handle multi-tenancy. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/better-sql-functions-in-postgresql-v14/ https://jkatz05.com/post/postgres/postgres-begin-atomic/ https://www.pgmustard.com/blog/using-postgres-buffers-for-query-optimization https://www.citusdata.com/blog/2022/07/28/debugging-postgres-autovacuum-problems-13-tips/ https://www.thenile.dev/blog/multi-tenant-rls https://pganalyze.com/blog/5mins-postgres-row-level-security-bypassrls-security-invoker-views-leakproof-functions https://www.timescale.com/blog/state-of-postgresql-2022-13-tools-that-arent-psql/ https://www.youtube.com/c/percona/videos https://www.crunchydata.com/blog/native-partitioning-with-postgres https://www.endpointdev.com/blog/2022/07/running-postgres

In this episode of Scaling Postgres, we discuss /gexec available in psql, how to delete duplicates using a window function, a new Postgres podcast and a performance talk about going from puny to powerful. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/gexec-psql-postgresql-poweruser-practice/ https://www.postgresql.org/docs/14/functions-string.html#FUNCTIONS-STRING-FORMAT https://sqlfordevs.io/delete-duplicate-rows https://www.pgmustard.com/blog/postgres-fm-a-new-postgres-podcast https://www.youtube.com/watch?v=CIYbpYKrX8Y https://pganalyze.com/blog/5mins-postgres-waiting-for-postgres-16-subquery-alias-optional https://www.youtube.com/watch?v=mJw3al4Ms2o https://ardentperf.com/2022/07/16/postgresql-performance-puzzle/ https://postgresql.life/post/laura_ricci/ https://www.rubberduckdevshow.com/episodes/51-andrew-mason-collin-jilbert-ruby-radar/  

In this episode of Scaling Postgres, we discuss how too many columns in a table can affect performance, at what point BRIN indexes win over btree, and Postgres 15 supporting unique nulls and parallel distinct queries. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/column-order-in-postgresql-does-matter/ https://www.crunchydata.com/blog/postgres-indexing-when-does-brin-win https://blog.rustprooflabs.com/2022/07/postgres-15-unique-improvement-with-null https://www.percona.com/blog/introducing-postgresql-15-working-with-distinct/ https://pganalyze.com/blog/5mins-postgres-unique-constraint-null-parallel-distinct https://postgrespro.com/blog/pgsql/5969637 https://coroot.com/blog/pg-agent https://www.timescale.com/blog/state-of-postgresql-how-to-contribute-to-postgresql-and-the-community/ https://shekhargulati.com/2022/07/08/my-notes-on-gitlabs-postgres-schema-design/ https://pgstef.github.io/2022/07/12/patroni_an

In this episode of Scaling Postgres, we discuss the process of decomposing your database, how to search text in Postgres quickly, how to understand the output of pg_stat_activity and how to use window functions to solve a use case. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://about.gitlab.com/blog/2022/06/02/splitting-database-into-main-and-ci/ https://gitlab.com/groups/gitlab-org/-/epics/6168 https://docs.gitlab.com/ee/development/database/multiple_databases.html https://www.rubberduckdevshow.com/episodes/49-fast-text-search-in-postgres/ https://www.depesz.com/2022/07/05/understanding-pg_stat_activity/ https://www.cybertec-postgresql.com/en/handling-bonus-programs-in-sql/ https://www.crunchydata.com/blog/choosing-a-postgresql-number-format https://www.crunchydata.com/blog/data-to-go-postgres-logical-replication https://www.percona.com/blog/postgresql-replication-and-conflicts/ https://www.timescale.com/blog/state-of-postgresql-2022-fi

In this episode of Scaling Postgres, we discuss psql shortcuts, how to debug deadlocks, how to find & stop queries and how to understand and use SCRAM authentication. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-15-beta-2-released-2479/ https://www.crunchydata.com/blog/postgres-locking-when-is-it-concerning https://www.percona.com/blog/importance-of-postgresql-vacuum-tuning-and-custom-scheduled-vacuum-job/ https://www.cybertec-postgresql.com/en/transaction-anomalies-with-select-for-update/ https://andreas.scherbaum.la/blog/archives/1117-ctid-and-other-PostgreSQL-table-internals.html https://www.postgresql.fastware.com/blog/introducing-publication-row-filters https://pganalyze.com/blog/5mins-postgres-tuning-statistics-target-selectivity-spatial-indexes https://andreas.scherbaum.la/blog/archives/1118-2-+-2-4-16.html https://supabase.com/blog/2022/06/28/partial-postgresql-data-dumps-with-rls ht

In this episode of Scaling Postgres, we discuss psql shortcuts, how to debug deadlocks, how to find & stop queries and how to understand and use SCRAM authentication. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://elephas.io/psql-helper-managing-connections-and-simplifying-queries/ https://www.cybertec-postgresql.com/en/debugging-deadlocks-in-postgresql/ https://adamj.eu/tech/2022/06/20/how-to-find-and-stop-running-queries-on-postgresql/ https://www.youtube.com/watch?v=whvXiAa_T6A https://andreas.scherbaum.la/blog/archives/1116-PostgreSQL-Upgrades-are-hard!.html https://fluca1978.github.io/2022/06/23/PostgreSQLOrdinality.html https://www.crunchydata.com/blog/indexes-selectivity-and-statistics https://blog.rustprooflabs.com/2022/06/h3-indexes-on-postgis-data https://pganalyze.com/blog/5mins-postgres-fuzzy-text-search-case-insensitive-ICU-collations https://www.timescale.com/blog/how-we-made-data-aggregation-better-and-faster-on-postgre

In this episode of Scaling Postgres, we discuss a PG 14.4 release, how to skip transactions with logical replication, how to use schemas and Citus going fully open source. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-144-released-2470/ https://www.postgresql.fastware.com/blog/addressing-replication-conflicts-using-alter-subscription-skip https://aaronoellis.com/articles/using-postgres-schemas https://www.citusdata.com/blog/2022/06/17/citus-11-goes-fully-open-source/ https://www.cybertec-postgresql.com/en/case-insensitive-pattern-matching-in-postgresql/ https://blog.brendanscullion.com/postgresql-text-search https://www.crunchydata.com/blog/quick-and-easy-postgres-data-compare https://sqlfordevs.io/uuid-prevent-enumeration-attack https://andreas.scherbaum.la/blog/archives/1113-My-PostgreSQL-database-is-empty!.html https://pganalyze.com/blog/5mins-postgres-reducing-aws-aurora-io-costs-table-partitio

In this episode of Scaling Postgres, we discuss a Postgres 14 out-of-cycle release next week, whether you should upgrade, the state of Postgres survey and a tool for automatic indexing. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode:

In this episode of Scaling Postgres, we discuss a severe bug with index creation in Postgres 14, view permissions options, dealing with logical replication conflicts and looking under the hood of AlloyDB. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://pganalyze.com/blog/5mins-postgres-14-reindex-create-index-concurrently-bug-amcheck-extension https://git.postgresql.org/gitweb/?p=postgresql.git;a=commit;h=042b584c7f7d6216c54359c0ee0f613ba3b3d9c2 https://www.postgresql.org/docs/14/amcheck.html https://www.cybertec-postgresql.com/en/view-permissions-and-row-level-security-in-postgresql/ https://www.postgresql.fastware.com/blog/how-to-handle-logical-replication-conflicts-in-postgresql https://cloud.google.com/blog/products/databases/alloydb-for-postgresql-intelligent-scalable-storage https://cloud.google.com/blog/products/databases/alloydb-for-postgresql-columnar-engine https://www.enterprisedb.com/blog/using-virtual-environment-plpython3-post

In this episode of Scaling Postgres, we discuss mistakes you can make when doing schema changes, how best to cancel statements, looking into pg_rman, and looking at a pedantry removal. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://postgres.ai/blog/20220525-common-db-schema-change-mistakes https://www.cybertec-postgresql.com/en/cancel-hanging-postgresql-query/ https://www.highgo.ca/2022/05/27/how-to-do-backup-and-restore-for-postgresql-using-pg_rman/ https://mydbanotebook.org/post/pedantry/ https://pganalyze.com/blog/5mins-postgres-15-speeding-up-sort-performance https://www.timescale.com/blog/opentelemetry-where-sql-is-better-than-the-original/ https://www.crunchydata.com/blog/exporter2perfdata-connecting-postgres-monitoring-tools https://postgresql.life/post/masahiko_sawada/ https://www.rubberduckdevshow.com/episodes/43-typed-or-untyped-ruby/  

In this episode of Scaling Postgres, we discuss the Beta 1 release of Postgres 15, optimizing a query by breaking it down, improvements to sort performance and prepared statements for partitioned tables. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-15-beta-1-released-2453/ https://www.crunchydata.com/blog/postgres-query-optimization-left-join-vs-union-all https://www.citusdata.com/blog/2022/05/19/speeding-up-sort-performance-in-postgres-15/ https://amitlan.com/2022/05/16/param-query-partition-woes.html https://pganalyze.com/blog/5mins-postgres-partition-pruning-prepared-statements-generic-vs-custom-query-plans https://www.ongres.com/blog/solving-postgresql-statistic-corruption/ https://www.craigkerstiens.com/2022/05/18/unfinished-business-with-postgres/ https://ptc-it.de/collate-docker-postgres-bug/ https://geopet85.github.io/2022/05/15/rds-partitioned-tables-export.html https://www.crunchydata.c

In this episode of Scaling Postgres, we discuss parallel server-side backup compression, IO in Postgres, parquet files and the new pg_stat_monitor extension. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.postgresql.org/about/news/postgresql-143-137-1211-1116-and-1021-released-2449/ https://cloud.google.com/blog/products/databases/introducing-alloydb-for-postgresql https://www.youtube.com/watch?v=o5pKfH0Tonw https://www.cybertec-postgresql.com/en/time-zone-management-in-postgresql/ https://www.enterprisedb.com/blog/pg-phriday-what-you-should-know-about-basic-postgres-authentication https://www.crunchydata.com/blog/the-vectors-of-database-encryption https://www.crunchydata.com/blog/demystifying-database-performance-for-developers https://pganalyze.com/blog/5mins-postgres-for-app-developers-tables-indexes https://kiranrao.ca/2022/05/04/zero-downtime-migrations.html https://www.se-radio.net/2022/05/episode-511-ant-wilson-on-supabase-post

In this episode of Scaling Postgres, we discuss parallel server-side backup compression, IO in Postgres, parquet files and the new pg_stat_monitor extension. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: http://rhaas.blogspot.com/2022/05/parallel-server-side-backup-compression.html https://www.youtube.com/watch?v=3Oj7fBAqVTw https://www.crunchydata.com/blog/parquet-and-postgres-in-the-data-lake https://www.percona.com/blog/understand-your-postgresql-workloads-better-with-pg_stat_monitor/ https://www.timescale.com/blog/point-in-time/ https://www.cybertec-postgresql.com/en/postgresql-15-using-merge-in-sql/ https://www.highgo.ca/2022/04/30/upcoming-feature-in-postgresql-15-merge-statement/ https://andreas.scherbaum.la/blog/archives/1110-Its-beer-oclock-for-PostgreSQL.html https://pganalyze.com/blog/5mins-postgres-15-beta1-incremental-materialized-views-pg-ivm https://www.highgo.ca/2022/05/06/atomic-commit-with-2-phase-commit-in-fdw-distributed-setu

In this episode of Scaling Postgres, we discuss ways you can optimize the planner, becoming part of the Postgres Community, considerations for large databases and mapping three different types of spatial data. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.aha.io/engineering/articles/optimizing-with-the-postgresql-deterministic-query-planner https://pganalyze.com/blog/5mins-postgres-tuning-deterministic-query-planner-extended-statistics-join-collapse-limits https://www.timescale.com/blog/the-future-of-community-in-light-of-babelfish/ https://www.percona.com/blog/working-with-large-postgresql-databases/ https://blog.rustprooflabs.com/2022/04/postgis-h3-intro https://selectfrom.dev/mapping-fracking-wells-in-colorado-with-postgresql-db17fc3b8812 https://www.cybertec-postgresql.com/en/analyzing-historical-flight-data-with-mobilitydb/ https://www.highgo.ca/2022/04/29/global-deadlock-in-a-distributed-database-cluster/ https://www.endpointde

In this episode of Scaling Postgres, we discuss dealing with slow queries, upgrading a 4TB database, running Postgres on ZFS and handling a storage stampede. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://pgdash.io/blog/slow-queries-postgres.html https://retool.com/blog/how-we-upgraded-postgresql-database/ https://lackofimagination.org/2022/04/our-experience-with-postgresql-on-zfs/ https://www.enterprisedb.com/blog/steady-storage-stampede https://tech.instacart.com/the-vanishing-thread-and-postgresql-tcp-connection-parameters-93afc0e1208c https://www.cybertec-postgresql.com/en/find-and-fix-a-missing-postgresql-index/ https://www.percona.com/blog/postgresql-14-b-tree-index-reduced-bloat-with-bottom-up-deletion/ https://pganalyze.com/blog/5mins-postgres-hot-updates-vs-bottom-up-index-deletion https://www.youtube.com/watch?v=oJj-pltxBUM https://www.postgresql.fastware.com/blog/two-phase-commits-for-logical-replication-publications-subscription

In this episode of Scaling Postgres, we discuss working with multiranges, missing Postgres metrics, PostGIS for newbies and videos from CitusCon. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/multiranges-in-postgresql-14/ https://coroot.com/blog/pg-missing-metrics https://blog.crunchydata.com/blog/postgis-for-newbies https://www.youtube.com/playlist?list=PLlrxD0HtieHjSzUZYCMvqffEU5jykfPTd#cituscon2022 https://pganalyze.com/blog/5mins-postgres-sql-json-table-postgres15 https://yugonagata-pgsql.blogspot.com/2022/04/pgivm-postgresql-extension-providing.html https://www.postgresql.org/about/news/update-on-the-trademark-actions-against-the-postgresql-community-2437/ https://pgstef.github.io/2022/04/15/pgbackrest_multi-repositories_tips_and_tricks.html https://postgresql.life/post/jehan-guillaume_de_rorthais/ https://www.rubberduckdevshow.com/episodes/38-small-projects/  

In this episode of Scaling Postgres, we discuss how to bin date times, search every field of a table, use JSON_TABLE and methods for a client to pick the primary database. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.cybertec-postgresql.com/en/postgresql-mapping-timestamps-date_bin/ https://www.depesz.com/2022/04/05/dba-tips-n-tricks-search-in-every-field-in-a-table/ https://www.depesz.com/2022/04/06/waiting-for-postgresql-15-json_table/ https://www.postgresql.org/docs/14/libpq-connect.html#LIBPQ-CONNECT-TARGET-SESSION-ATTRS https://www.cybertec-postgresql.com/en/new-target_session_attrs-settings-for-high-availability-and-scaling-in-postgresql-v14/ https://www.postgresql.fastware.com/blog/enhancement-to-target-session-attrs https://pganalyze.com/blog/5mins-postgres-tuning-vacuum-autovacuum https://www.endpointdev.com/blog/2022/04/shapefiles-postgis/ https://dev.to/yugabyte/how-to-enable-ssl-for-postgres-connections-5321 https://pos

In this episode of Scaling Postgres, we discuss how to optimize vacuum, how to drop roles, the new merge command in PG15 and a pgbouncer tutorial. Subscribe at https://www.scalingpostgres.com to get notified of new episodes. Links for this episode: https://www.enterprisedb.com/blog/pg-phriday-tidying-vacuum https://www.cybertec-postgresql.com/en/drop-role-or-drop-user-postgresql/ https://www.depesz.com/2022/03/31/waiting-for-postgresql-15-add-support-for-merge-sql-command/ https://www.postgresql.org/docs/devel/sql-merge.html https://www.youtube.com/watch?v=ddKm7a7xOpk https://www.depesz.com/2022/03/31/waiting-for-postgresql-15-sql-json-constructors/ https://www.depesz.com/2022/04/01/waiting-for-postgresql-15-is-json-predicate/ https://www.depesz.com/2022/04/01/waiting-for-postgresql-15-sql-json-query-functions/ https://www.highgo.ca/2022/03/31/parallel-commit-in-postgres-fdw/ https://pganalyze.com/blog/how-postgres-chooses-index https://postgrespro.com/blog/pgsql/5969403 https://www.citusdata.com/blog