- Update the default configuration to allow direct authentication from the local machine:
sed -i 's/^local\s\+all\s\+all\s\+ident/local all all trust/g' /etc/postgresql/8.4/main/pg_hba.conf
- Same as above but for local IPv4 and IPv6 connexions:
sed -i 's/^host\s\+all\s\+all\s\+\(.*\)\s\+md5/host all all \1 trust/g' /etc/postgresql/8.4/main/pg_hba.conf
- List databases:
psql --list -U kevin
- Create a new
kevin_dbdatabase with thekevinuser:createdb -U kevin kevin_db
- Remove the database we created above:
dropdb kevin_db -U kevin
- To connect to a particular database:
psql -d database_id
- Show us how a table of a specific database can be recreated:
pg_dump my_database --schema-only --table=my_table
- Dump a database in a compressed format:
pg_dump my_database -v --format=c --file=/var/lib/postgresql/my_database-db-2011-12-19.dump
- Restore a compressed dump:
pg_restore -U my_user -d my_database /var/lib/postgresql/my_database-db-2011-12-19.dump
- Import an SQL file to a database:
psql --username kevin -d kevin_db < ./database_dump.sql
- Search if
kevinis a PostgreSQL user:sudo -u postgres psql --tuples-only --no-align --command "SELECT usename FROM pg_user;" | grep --quiet 'kevin' && echo 'User found !' || echo 'User not found !'
- Set the owner of a database:
ALTER DATABASE db_id OWNER TO user_id;
- Set the owner of all tables from the
MY_DB_IDdatabase toMY_DB_USER(source):for tbl in `psql -qAt -c "SELECT tablename FROM pg_tables WHERE schemaname = 'public';" MY_DB_ID` ; do psql -c "ALTER TABLE $tbl OWNER TO MY_DB_USER" MY_DB_ID ; done
- Same as above but for sequences and views:
for tbl in `psql -qAt -c "SELECT sequence_name FROM information_schema.sequences WHERE sequence_schema = 'public';" MY_DB_ID` ; do psql -c "ALTER TABLE $tbl OWNER TO MY_DB_USER" MY_DB_ID ; done for tbl in `psql -qAt -c "SELECT table_name FROM information_schema.views WHERE table_schema = 'public';" MY_DB_ID` ; do psql -c "ALTER TABLE $tbl OWNER TO MY_DB_USER" MY_DB_ID ; done
- Remove from a table all rows older than a month:
sudo -u postgres psql -d database_id --command "DELETE FROM smile_log WHERE log_date > current_date - interval '1 month';"
- Monitor queries being run in real time:
watch -n 1 'sudo -u postgres psql --tuples-only --command "SELECT datname, procpid, date_trunc(\$\$second\$\$, age(current_timestamp, xact_start)), current_query FROM pg_stat_activity;"'
- Disable all triggers of a table, excluding triggers that are used to implement foreign key constraints:
ALTER TABLE table_id DISABLE TRIGGER ALL;
- List all constraints of your database (source):
SELECT tc.constraint_name, tc.constraint_type, tc.table_name, kcu.column_name, tc.is_deferrable, tc.initially_deferred, rc.match_option AS match_type, rc.update_rule AS on_update, rc.delete_rule AS on_delete, ccu.table_name AS references_table, ccu.column_name AS references_field FROM information_schema.table_constraints tc LEFT JOIN information_schema.key_column_usage kcu ON tc.constraint_catalog = kcu.constraint_catalog AND tc.constraint_schema = kcu.constraint_schema AND tc.constraint_name = kcu.constraint_name LEFT JOIN information_schema.referential_constraints rc ON tc.constraint_catalog = rc.constraint_catalog AND tc.constraint_schema = rc.constraint_schema AND tc.constraint_name = rc.constraint_name LEFT JOIN information_schema.constraint_column_usage ccu ON rc.unique_constraint_catalog = ccu.constraint_catalog AND rc.unique_constraint_schema = ccu.constraint_schema AND rc.unique_constraint_name = ccu.constraint_name WHERE lower(tc.constraint_type) in ('foreign key'); - And finally, here is a list of great monitoring one-liners.
Monthly Archives: October 2011
Installation Guide for a full-featured Debian server
Here is a collection of articles I wrote during the past year. Together they form a guide that will let you setup a full-featured Debian server. All of these tutorials are based on the recent work I did to setup my personal server on Debian Squeeze.
These articles are independent with each other, meaning you can pick the one your interested in to customize your server and ignore the others.
- Setup SMART monitoring tool for HDDs.
- Setup Nut to manage the UPS.
- Setup Duplicity and Amazon S3 for cloud-based backups.
- Setup Exim to relay mails via Gmail.
- Setup cron-apt to keep our distribution up to date.
- Add a fail2ban deamon.
- Setup Munin to monitor our machine.
- Basic setup of Nginx + PHP-FPM + MySQL web stack.
- Optimizing Nginx + PHP-FPM + MySQL for performances.
- Setup PHP APC op-code cache.
- Install haveged to get lots of entropy.
- Setup a WebDAVs server with Lighttpd.
- Setup Mailman + Nginx + Exim for mailing-lists.
- Mailman mailing-list migration and merging.
How-to merge Mailman mailing-lists
Let’s say I have an old inactive mailing list (which ID is old-ml) for which I want to merge its archives to another one (called active-ml).
To do so, I have to merge the two mbox files holding all mails since the creation of these mailing-lists. I first tried to use cat to concatenate the two mbox files be it didn’t work.
Luckily, I found a Python script to merge 2 mbox files while sorting all mails by date. Here is how I uses it:
$ cd /var/lib/mailman/archives/private $ wget http://mail.python.org/pipermail/mailman-users/attachments/20080322/80455064/attachment.txt --output-document=mbmerge.py $ python ./mbmerge.py ./old-ml.mbox/old-ml.mbox ./active-ml.mbox/active-ml.mbox > ./active-ml.mbox/active-ml.mbox.new
Then I switched the current mbox with the one generated above and asked mailman to regenerate the static HTML archives:
$ cd /var/lib/mailman/archives/private/active-ml.mbox/ $ mv active-ml.mbox active-ml.mbox.backup $ mv active-ml.mbox.new active-ml.mbox $ chown list:list active-ml.mbox* $ /usr/lib/mailman/bin/arch --wipe active-ml
Of course this will only merge mail archives. You still have to merge your old mailing lists parameters (including membership) manually.
At last, when everything is clean to you, you can safely remove your old mailing-list:
$ rmlist -a old-ml $ /var/lib/mailman/bin/genaliases
Mailman migration
Last week I detailed how I configured Mailman with Exim and Nginx on a Debian Squeeze. Here are some more notes on how I migrated my mailing lists from my old server (Lenny with Mailman 2.1.11) to the new Mailman installation (Squeeze with Mailman 2.1.13).
First, I remove the default mailman meta-list as I will retrieve the one from the old server:
$ /etc/init.d/mailman stop $ rmlist -a mailman $ /var/lib/mailman/bin/genaliases
Then I copy mailing-list data from the old server to the new:
$ rsync --progress -vrae "ssh -C" /var/lib/mailman/lists root@new.example.com:/var/lib/mailman/ $ rsync --progress -vrae "ssh -C" /var/lib/mailman/archives root@new.example.com:/var/lib/mailman/ $ rsync --progress -vrae "ssh -C" /var/lib/mailman/data root@new.example.com:/var/lib/mailman/
Back to our new server, fix some rights, check all lists are there, and run the automatic update:
$ chown -R list:list /var/lib/mailman/ $ /etc/init.d/mailman start $ list_lists $ /var/lib/mailman/bin/update
Now let Mailman check its databases and fix permission:
$ check_db -a -v $ check_perms -f -v
At this point you may get this error in your /var/log/exim4/mainlog:
2011-09-13 10:06:09 failed to expand condition "${lookup{$local_part@$domain}lsearch{/var/lib/mailman/data/virtual-mailman}{1}{0}}" for mailman_router router: failed to open /var/lib/mailman/data/virtual-mailman for linear search: Permission denied (euid=101 egid=103)
This can be fixed with (source):
$ chgrp Debian-exim /var/lib/mailman/data/virtual-mailman
You may also encounter this error:
2011-09-13 10:06:09 H=mail-xxx-xxxx.google.com [209.85.000.000] F=<kevin@example.com> rejected RCPT <kev-test@lists.example.com>: Unrouteable address
In this case regenerating Mailman aliases should fix the issue:
$ /var/lib/mailman/bin/genaliases
By the way, to test that Exim is routing mails as expected, your can use the following command:
$ exim -bt kev-test@lists.example.com R: system_aliases for kev-test@lists.example.com R: mailman_router for kev-test@lists.example.com kev-test@lists.example.com router = mailman_router, transport = mailman_transport
Last problem I had was mails did not reached my server. Everytime I send something from Gmail to a list, I got back error mails saying this:
Technical details of permanent failure:
Google tried to deliver your message, but it was rejected by the recipient domain. We recommend contacting the other email provider for further information about the cause of this error. The error that the other server returned was: 550 550 relay not permitted (state 14).
I fixed this issue by updating my SPF record on the example.com domain from:
v=spf1 a mx ~all
to:
v=spf1 a mx ptr ~all