PgCon 2019 Developer Unconference
From PostgreSQL wiki
Jump to navigationJump to searchUnconference schedule by Time and Room
| Time | DMS 1160 | DMS 1110 | DMS 1120 |
|---|---|---|---|
| Tue 11:15-12:15 | Zed Store | Test Coverage | Parallel Grouping Sets |
| Tue 12:15-13:15 | Lunch | ||
| Tue 13:15-14:15 | DirectIO & AsyncIO | PostGIS 3.0 | Locale apocalypse, collation, ICU stuff |
| Tue 14:15-14:30 | Coffee Break | ||
| Tue 14:30-15:30 | Problems / Possible solutions for high volume / latency links w/replication | Fault Injection Framework | SLRU to Buffer Cache but without on disk changes |
| Tue 15:30-16:00 | Break | ||
| Tue 16:00-17:00 | WAL Archiving extensions | Constrain Session memory | SQL standard stuff |
Notes/Minutes
Test Coverage
- We talked through various details of the existing coverage report.
- David Steele talked about testing in pgbackrest project: https://github.com/pgbackrest/pgbackrest/blob/master/test/src/test.c
- Interested in fault injection to get coverage for hard to test code paths.
- Actions:
- Write more tests. Everyone is welcome.
- Let's get all function coverage scores to green.
- Look into reactivating branch coverage.
- Look into generating coverage diffs via cfbot.
Locale apocalypse
- Upgrade from glibc 2.27 to 2.28 will break indexes for many users.
- Details on this page: Locale_data_changes
- Point users to that page.
- Update that page with information on different OS versions over time.
- Plan: Send out a community announcement about this soon.
- Various technical mitigation options were discussed, but none seemed feasible to help existing users in the near future.
- Plan for PostgreSQL 13: Make ICU an option for the global collation. Make it the default in the more distant future.
- Concern: ICU might be slower. Should benchmark.
SQL standard stuff
- Peter Eisentraut reported that he has joined the SQL standard working group and gave some insights into its inner workings.
- slides of a presentation by Keith Hare that gives some information
- For any more questions, contact Peter directly.
- SQL:2020 will contain a built-in JSON data type; concerns about how this will interoperate with PostgreSQL's existing json/jsonb types; Peter and Oleg will investigate
Direct I/O & Async I/O
Overview of fsync problem
Goals:
- Better error handling
- Better performance
- More predictability
- Observibility
- Remove double-buffering
Tasks:
- Background writer needs to actually write pages out, as well as trying to maintain a set of "free" pages
- Write throughput
- Write combining
- Smarter buffer eviction
Fault Injection
Requirements
- system call error scenarios
- protocol errors
- hard to hit errors / fine-grained control
- easy to write/read
- deliberate corruption
- concurrency
- slow down execution
- stress testing (ie: XID burn)
- uncommon combos
- SQL combination testing
- fuzz testing
- granular sequencing
- Mutation testing
Notes
- Why isn't Linux Fault Injection implemented?
Frameworks / solutions
- pg_crash
- pg_panic
- Jeff Jane's framework
- pgBackRest's framework
- Dtrace can over-write returns calls of system calls
- pg_swap_pages
- ptrace
- Microsoft test framework
https://github.com/greenplum-db/gpdb/tree/master/gpcontrib/gp_inject_fault
- Add fault points to core
- Extension that allows for triggering faults via SQL
* Place of fault * Type of fault
- Extension communicates fault requests to core code via shared memory
