INFORMATICS

The Best

Backup & Restore a PostgreSQL Database

Star InactiveStar InactiveStar InactiveStar InactiveStar Inactive

Backup & Restore a PostgreSQL Database

Postgres provides to 3 fundamentally different ways of backing up your database. 

  • SQL Dump
  • File System Backup
  • Continuous Archiving

SQL Dump

All commands should be run as the postgres user.

sudo su - postgres

 

The SQL dump (pg_dump) is a command that generates a text file that contains necessary SQL instruction that will recreate the database. The dumped file can be fed back to the server to recreate the existing structure of the database, including the data itself.

pg_dump dbname > dbbackupfilename.sql


The pg_dump command can be used to perform database backups from any remote host that you have access to.
It's important to understand that the command needs access to all tables that you want to backup so it should be initiated by a database user with superuser privilege.


If your are working with a large database it useful to gizip the file as you dump it. You can gizip the dump by running the following command instead. pg_dump dbname | gzip > dbbackupfilename.gz


Restore PostgreSQL pg_dump To restore the pg_dump file we make use of the psql command psql dbname < dbbackupfilename If you want to make sure that the database restoration stops incase of an error related you can set the ON_ERROR_STOP flag. psql --set ON_ERROR_STOP=on dbname < dbbackupfilename If you are handling a large database you might have gzipped the dump to minimize file-size and transfer time. To restore a gzipped pg_dump file you can run the following command. gunzip -c filename.gz | psql dbname


2: File System Backup In some cases the SQL dump is not optimal for your situation. Another useful way to backup PostgreSQL is to do a file system backup that directly copies all the file that PostgreSQL uses to store data in the database.

tar -cf dbbackupfilename.tar /usr/local/pgsql/data It's important to understand that the file system-level backup imposes extra restrictions when compared with the pg_dump method. file system backups require the database to be turned off in order to generate usable backups file system backups only work for complete backup and restoration of an entire database cluster.


--------


Basic backup and restore commands # Backup a single database pg_dump db_name > db_backup.sql # Restore a single database psql db_name < db_backup.sql # Backup an entire postgres database cluster pg_dumpall > cluster_backup.sql # Restore an entire postgres database cluster psql -f cluster_backup.sql postgres Backup and Restore using compression # Backup a single database pg_dump db_name | gzip > db_backup.gz # Restore a single database gunzip -c db_backup.gz | psql db_name # Backup a database cluster pg_dumpall | gzip > cluster_backup.gz # Restore a database cluster gunzip -c cluster_backup.gz | psql postgres Backup and be able to restore individual tables # Backup a single database pg_dump -Fc db_name > db_backup.dmp # Restore a single database pg_restore -d db_name db_backup.dmp # Can use pg_dumpall to backup all global information # then use pg_dump to backup each database pg_dumpall > global_only_backup.sql --globals-only Restore the database and stop on errors psql db_name < db_backup.sql --set ON_ERROR_STOP=on After restore, vacuum and analyze tables vacuumdb -a -z Basic Backup script #!/bin/bash # # Takes a full backup of the database and stores it in the backup folder # Run this script as the postgres user # DATE=`date +%Y-%m-%d` echo `date` - Delete old backups find ~/backup/* -mtime +1 -delete echo `date` - Do a full postgres cluster dump pg_dumpall | gzip > ~/backup/db_cluster_dump_$DATE.gz echo `date` - Sync pg_backups with S3 # /usr/local/bin/aws s3 sync ~/backup s3://bucket_name/backup echo `date` - Sync postgres configuration files with S3 # /usr/local/bin/aws s3 sync /etc/postgresql/9.3/main s3://bucket_name/backup echo `date` - Backup complete Schedule the script using cron # Make sure script is executable by postgres chmod 770 backup_script.sh crontab -e # m h dom mon dow command 0 4 * * * ~/backup_script.sh > ~/backup_script.log 2>&1

pg restore manual

Star InactiveStar InactiveStar InactiveStar InactiveStar Inactive
pg restore manual:

pg_restore - restore a PostgreSQL database from an archive file created by pg_dump
pg_restore is a utility for restoring a PostgreSQL database from an archive created by pg_dump(1) in one of the non-plain-text formats. It will issue the commands necessary to reconstruct the database to the state it was in at the time it was saved.

pg_restore accepts the following command line arguments. filename Specifies the location of the archive file (or directory, for a directory-format archive) to be restored. If not specified, the standard input is used. -a, --data-only Restore only the data, not the schema (data definitions). Table data, large objects, and sequence values are restored, if present in the archive. This option is similar to, but for historical reasons not identical to, specifying --section=data. -c, --clean Clean (drop) database objects before recreating them. (This might generate some harmless error messages, if any objects were not present in the destination database.) -C, --create Create the database before restoring into it. If --clean is also specified, drop and recreate the target database before connecting to it. When this option is used, the database named with -d is used only to issue the initial DROP DATABASE and CREATE DATABASE commands. All data is restored into the database name that appears in the archive. -d dbname, --dbname=dbname Connect to database dbname and restore directly into the database. -e, --exit-on-error Exit if an error is encountered while sending SQL commands to the database. The default is to continue and to display a count of errors at the end of the restoration. -f filename, --file=filename Specify output file for generated script, or for the listing when used with -l. Default is the standard output. -F format, --format=format Specify format of the archive. It is not necessary to specify the format, since pg_restore will determine the format automatically. If specified, it can be one of the following: c, custom The archive is in the custom format of pg_dump. d, directory The archive is a directory archive. t, tar The archive is a tar archive. -i, --ignore-version A deprecated option that is now ignored. -I index, --index=index Restore definition of named index only. -j number-of-jobs, --jobs=number-of-jobs Run the most time-consuming parts of pg_restorethose which load data, create indexes, or create constraintsusing multiple concurrent jobs. This option can dramatically reduce the time to restore a large database to a server running on a multiprocessor machine. Each job is one process or one thread, depending on the operating system, and uses a separate connection to the server. The optimal value for this option depends on the hardware setup of the server, of the client, and of the network. Factors include the number of CPU cores and the disk setup. A good place to start is the number of CPU cores on the server, but values larger than that can also lead to faster restore times in many cases. Of course, values that are too high will lead to decreased performance because of thrashing. Only the custom and directory archive formats are supported with this option. The input must be a regular file or directory (not, for example, a pipe). This option is ignored when emitting a script rather than connecting directly to a database server. Also, multiple jobs cannot be used together with the option --single-transaction. -l, --list List the contents of the archive. The output of this operation can be used as input to the -L option. Note that if filtering switches such as -n or -t are used with -l, they will restrict the items listed. -L list-file, --use-list=list-file Restore only those archive elements that are listed in list-file, and restore them in the order they appear in the file. Note that if filtering switches such as -n or -t are used with -L, they will further restrict the items restored. list-file is normally created by editing the output of a previous -l operation. Lines can be moved or removed, and can also be commented out by placing a semicolon (;) at the start of the line. See below for examples. -n namespace, --schema=schema Restore only objects that are in the named schema. This can be combined with the -t option to restore just a specific table. -O, --no-owner Do not output commands to set ownership of objects to match the original database. By default, pg_restore issues ALTER OWNER or SET SESSION AUTHORIZATION statements to set ownership of created schema elements. These statements will fail unless the initial connection to the database is made by a superuser (or the same user that owns all of the objects in the script). With -O, any user name can be used for the initial connection, and this user will own all the created objects. -P function-name(argtype [, ...]), --function=function-name(argtype [, ...]) Restore the named function only. Be careful to spell the function name and arguments exactly as they appear in the dump file's table of contents. -R, --no-reconnect This option is obsolete but still accepted for backwards compatibility. -s, --schema-only Restore only the schema (data definitions), not data, to the extent that schema entries are present in the archive. This option is the inverse of --data-only. It is similar to, but for historical reasons not identical to, specifying --section=pre-data --section=post-data. (Do not confuse this with the --schema option, which uses the word “schema” in a different meaning.) -S username, --superuser=username Specify the superuser user name to use when disabling triggers. This is relevant only if --disable-triggers is used. -t table, --table=table Restore definition and/or data of named table only. Multiple tables may be specified with multiple -t switches. This can be combined with the -n option to specify a schema. -T trigger, --trigger=trigger Restore named trigger only. -v, --verbose Specifies verbose mode. -V, --version Print the pg_restore version and exit. -x, --no-privileges, --no-acl Prevent restoration of access privileges (grant/revoke commands). -1, --single-transaction Execute the restore as a single transaction (that is, wrap the emitted commands in BEGIN/COMMIT). This ensures that either all the commands complete successfully, or no changes are applied. This option implies --exit-on-error. --disable-triggers This option is relevant only when performing a data-only restore. It instructs pg_restore to execute commands to temporarily disable triggers on the target tables while the data is reloaded. Use this if you have referential integrity checks or other triggers on the tables that you do not want to invoke during data reload. Presently, the commands emitted for --disable-triggers must be done as superuser. So you should also specify a superuser name with -S or, preferably, run pg_restore as a PostgreSQL superuser. --no-data-for-failed-tables By default, table data is restored even if the creation command for the table failed (e.g., because it already exists). With this option, data for such a table is skipped. This behavior is useful if the target database already contains the desired table contents. For example, auxiliary tables for PostgreSQL extensions such as PostGIS might already be loaded in the target database; specifying this option prevents duplicate or obsolete data from being loaded into them. This option is effective only when restoring directly into a database, not when producing SQL script output. --no-security-labels Do not output commands to restore security labels, even if the archive contains them. --no-tablespaces Do not output commands to select tablespaces. With this option, all objects will be created in whichever tablespace is the default during restore. --section=sectionname Only restore the named section. The section name can be pre-data, data, or post-data. This option can be specified more than once to select multiple sections. The default is to restore all sections. The data section contains actual table data as well as large-object definitions. Post-data items consist of definitions of indexes, triggers, rules and constraints other than validated check constraints. Pre-data items consist of all other data definition items. --use-set-session-authorization Output SQL-standard SET SESSION AUTHORIZATION commands instead of ALTER OWNER commands to determine object ownership. This makes the dump more standards-compatible, but depending on the history of the objects in the dump, might not restore properly. -?, --help Show help about pg_restore command line arguments, and exit. pg_restore also accepts the following command line arguments for connection parameters: -h host, --host=host Specifies the host name of the machine on which the server is running. If the value begins with a slash, it is used as the directory for the Unix domain socket. The default is taken from the PGHOST environment variable, if set, else a Unix domain socket connection is attempted. -p port, --port=port Specifies the TCP port or local Unix domain socket file extension on which the server is listening for connections. Defaults to the PGPORT environment variable, if set, or a compiled-in default. -U username, --username=username User name to connect as. -w, --no-password Never issue a password prompt. If the server requires password authentication and a password is not available by other means such as a .pgpass file, the connection attempt will fail. This option can be useful in batch jobs and scripts where no user is present to enter a password. -W, --password Force pg_restore to prompt for a password before connecting to a database. This option is never essential, since pg_restore will automatically prompt for a password if the server demands password authentication. However, pg_restore will waste a connection attempt finding out that the server wants a password. In some cases it is worth typing -W to avoid the extra connection attempt. --role=rolename Specifies a role name to be used to perform the restore. This option causes pg_restore to issue a SET ROLErolename command after connecting to the database. It is useful when the authenticated user (specified by -U) lacks privileges needed by pg_restore, but can switch to a role with the required rights. Some installations have a policy against logging in directly as a superuser, and use of this option allows restores to be performed without violating the policy.


EXAMPLES
Assume we have dumped a database called mydb into a custom-format dump file: $ pg_dump -Fc mydb > db.dump To drop the database and recreate it from the dump: $ dropdb mydb $ pg_restore -C -d postgres db.dump The database named in the -d switch can be any database existing in the cluster; pg_restore only uses it to issue the CREATE DATABASE command for mydb. With -C, data is always restored into the database name that appears in the dump file. To reload the dump into a new database called newdb: $ createdb -T template0 newdb $ pg_restore -d newdb db.dump Notice we don't use -C, and instead connect directly to the database to be restored into. Also note that we clone the new database from template0 not template1, to ensure it is initially empty. To reorder database items, it is first necessary to dump the table of contents of the archive: $ pg_restore -l db.dump > db.list

1. Restore a database with psql command Recently, one of our customers approached us to restore the Postgres database from files. His database backup was a plaintext file containing SQL scripts. For restoring, our Support Engineers run the psql command. psql -U user_db user_name < dump_db.sql Here, user_db and user_name are the database user and database name respectively. And, dump_db.sql is the name of the backup file. This command effectively restored the Postgres database from files. 2. Restore a database with pg_restore command Similarly, another method to restore the Postgres database is using the pg_restore command. If the backup file created by pg_dump is of custom, directory or archive format, we use the command. pg_restore -d user_db /path/to/your/file/dump_db.tar -c -U user_name Here, -c prompts to drop the database before recreating it. By default, pg_restore has various options similar to this.

Migrating PostgreSQL files to new server

Star InactiveStar InactiveStar InactiveStar InactiveStar Inactive

Migrating PostgreSQL files to new server

Install the latest minor release of the same PostgreSQL major version - e.g. if you were using 9.4.1 before, install 9.4.26. Make sure it's the same architecture - i.e. if you were using a 32-bit PostgreSQL before you must install 32-bit PostgreSQL now.

  • Stop the service (postgresql) using the services control panel (services.msc).
  • Path in services - "C:\Program Files\PostgreSQL\9\bin\pg_ctl.exe" runservice -N "postgresql-x64-9" -D "D:\PostgreSQL\9\data" -w do not
  • Modify the configuration to the following with the sc command:  sc config postgresql-x64-9.4 binPath= "\"C:\Program Files\PostgreSQL\9.4\bin\pg_ctl.exe\" runservice -N \"postgresql-x64-9.4\" -D \"E:\pg_db\data\" -w"
  • Delete %PROGRAMFILES%\PostgreSQL\9.4\data.
  • Copy the old data directory to that location - copy C:\Program Files\PostgreSQL\9.4\data\ to E:\pg_db\data
  • Get properties on the data directory and in the security tab recursively grant full control to NETWORKSERVICE or pgsql -  - bad permissions will prevent the database from starting
  • Start the service using services.msc
  • immediately set up backups.

Search