README
author Radek Brich <brich.radek@ifortuna.cz>
Thu, 20 Mar 2014 16:33:34 +0100
changeset 95 6adcb7ee4517
parent 83 515fadd3d286
child 104 d8ff52a0390f
permissions -rw-r--r--
Update bigtables tool: Add size of indexes.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
0
eaae9539e910 Postgres tools.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
     1
=========
eaae9539e910 Postgres tools.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
     2
pgtoolkit
eaae9539e910 Postgres tools.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
     3
=========
eaae9539e910 Postgres tools.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
     4
76
3a41b351b122 Port pgconsole to Python3 + GTK3.
Radek Brich <radek.brich@devl.cz>
parents: 34
diff changeset
     5
Requirements
3a41b351b122 Port pgconsole to Python3 + GTK3.
Radek Brich <radek.brich@devl.cz>
parents: 34
diff changeset
     6
------------
3a41b351b122 Port pgconsole to Python3 + GTK3.
Radek Brich <radek.brich@devl.cz>
parents: 34
diff changeset
     7
3a41b351b122 Port pgconsole to Python3 + GTK3.
Radek Brich <radek.brich@devl.cz>
parents: 34
diff changeset
     8
 * Python 3.2+
3a41b351b122 Port pgconsole to Python3 + GTK3.
Radek Brich <radek.brich@devl.cz>
parents: 34
diff changeset
     9
 * Psycopg2
83
515fadd3d286 Add dependency on pycolib. Move common modules to pycolib. Add example table schema for meta DB.
Radek Brich <radek.brich@devl.cz>
parents: 76
diff changeset
    10
 * pycolib <http://hg.devl.cz/pycolib>
76
3a41b351b122 Port pgconsole to Python3 + GTK3.
Radek Brich <radek.brich@devl.cz>
parents: 34
diff changeset
    11
3a41b351b122 Port pgconsole to Python3 + GTK3.
Radek Brich <radek.brich@devl.cz>
parents: 34
diff changeset
    12
Optional:
83
515fadd3d286 Add dependency on pycolib. Move common modules to pycolib. Add example table schema for meta DB.
Radek Brich <radek.brich@devl.cz>
parents: 76
diff changeset
    13
 * for MyManager:
76
3a41b351b122 Port pgconsole to Python3 + GTK3.
Radek Brich <radek.brich@devl.cz>
parents: 34
diff changeset
    14
   * MySQLdb 1.2.3
3a41b351b122 Port pgconsole to Python3 + GTK3.
Radek Brich <radek.brich@devl.cz>
parents: 34
diff changeset
    15
   * see extras for Python3 patch
83
515fadd3d286 Add dependency on pycolib. Move common modules to pycolib. Add example table schema for meta DB.
Radek Brich <radek.brich@devl.cz>
parents: 76
diff changeset
    16
 * for pgconsole.py:
76
3a41b351b122 Port pgconsole to Python3 + GTK3.
Radek Brich <radek.brich@devl.cz>
parents: 34
diff changeset
    17
   * GTK3 + Python bindings + Cairo + GtkSource
3a41b351b122 Port pgconsole to Python3 + GTK3.
Radek Brich <radek.brich@devl.cz>
parents: 34
diff changeset
    18
   * packages in Debian: python3-gi python3-gi-cairo gir1.2-gtk-3.0 gir1.2-gtksource-3.0
83
515fadd3d286 Add dependency on pycolib. Move common modules to pycolib. Add example table schema for meta DB.
Radek Brich <radek.brich@devl.cz>
parents: 76
diff changeset
    19
 * for browser.py:
76
3a41b351b122 Port pgconsole to Python3 + GTK3.
Radek Brich <radek.brich@devl.cz>
parents: 34
diff changeset
    20
   * tuikit (http://hg.devl.cz/tuikit)
3a41b351b122 Port pgconsole to Python3 + GTK3.
Radek Brich <radek.brich@devl.cz>
parents: 34
diff changeset
    21
0
eaae9539e910 Postgres tools.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
    22
General Python Modules
eaae9539e910 Postgres tools.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
    23
----------------------
eaae9539e910 Postgres tools.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
    24
eaae9539e910 Postgres tools.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
    25
pgmanager.py - Database connection manager (pooling etc.)
eaae9539e910 Postgres tools.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
    26
pgbrowser.py - Schema browser module
eaae9539e910 Postgres tools.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
    27
pgdiff.py - Schema diff tool
eaae9539e910 Postgres tools.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
    28
eaae9539e910 Postgres tools.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
    29
Graphical and Command Line Tools
eaae9539e910 Postgres tools.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
    30
--------------------------------
eaae9539e910 Postgres tools.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
    31
eaae9539e910 Postgres tools.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
    32
pgbrowser-gtk.py - Database browser - graphical interface (GTK)
eaae9539e910 Postgres tools.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
    33
pgconsole-gtk.py - Query console - graphical interface (GTK)
eaae9539e910 Postgres tools.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
    34
pgdiff-cli.py - Command line diff tool
18
a9e12b7cc207 Fix MyManager. Add patch for MySQLdb and Python3.2. Add DelayedQuery.
Radek Brich <radek.brich@devl.cz>
parents: 0
diff changeset
    35
a9e12b7cc207 Fix MyManager. Add patch for MySQLdb and Python3.2. Add DelayedQuery.
Radek Brich <radek.brich@devl.cz>
parents: 0
diff changeset
    36
MySQL - mytoolkit
a9e12b7cc207 Fix MyManager. Add patch for MySQLdb and Python3.2. Add DelayedQuery.
Radek Brich <radek.brich@devl.cz>
parents: 0
diff changeset
    37
-----------------
a9e12b7cc207 Fix MyManager. Add patch for MySQLdb and Python3.2. Add DelayedQuery.
Radek Brich <radek.brich@devl.cz>
parents: 0
diff changeset
    38
a9e12b7cc207 Fix MyManager. Add patch for MySQLdb and Python3.2. Add DelayedQuery.
Radek Brich <radek.brich@devl.cz>
parents: 0
diff changeset
    39
Package mytoolkit contains PgManager drop-in replacement for MySQL,
a9e12b7cc207 Fix MyManager. Add patch for MySQLdb and Python3.2. Add DelayedQuery.
Radek Brich <radek.brich@devl.cz>
parents: 0
diff changeset
    40
MyManager. This may be useful if MySQL interoperability is needed
a9e12b7cc207 Fix MyManager. Add patch for MySQLdb and Python3.2. Add DelayedQuery.
Radek Brich <radek.brich@devl.cz>
parents: 0
diff changeset
    41
for some reason.
a9e12b7cc207 Fix MyManager. Add patch for MySQLdb and Python3.2. Add DelayedQuery.
Radek Brich <radek.brich@devl.cz>
parents: 0
diff changeset
    42
a9e12b7cc207 Fix MyManager. Add patch for MySQLdb and Python3.2. Add DelayedQuery.
Radek Brich <radek.brich@devl.cz>
parents: 0
diff changeset
    43
MyManager requires MySQLdb module, which works only with Python 2.5 - 2.7.
a9e12b7cc207 Fix MyManager. Add patch for MySQLdb and Python3.2. Add DelayedQuery.
Radek Brich <radek.brich@devl.cz>
parents: 0
diff changeset
    44
You need patch for Python 3.2 - see `extras/MySQL-python-1.2.3-python3.patch`.
a9e12b7cc207 Fix MyManager. Add patch for MySQLdb and Python3.2. Add DelayedQuery.
Radek Brich <radek.brich@devl.cz>
parents: 0
diff changeset
    45
a9e12b7cc207 Fix MyManager. Add patch for MySQLdb and Python3.2. Add DelayedQuery.
Radek Brich <radek.brich@devl.cz>
parents: 0
diff changeset
    46
MyManager may be merged into PgManager in future, but currently there
a9e12b7cc207 Fix MyManager. Add patch for MySQLdb and Python3.2. Add DelayedQuery.
Radek Brich <radek.brich@devl.cz>
parents: 0
diff changeset
    47
isn't any generally usable python module for MySQL. MySQLdb isn't developed
a9e12b7cc207 Fix MyManager. Add patch for MySQLdb and Python3.2. Add DelayedQuery.
Radek Brich <radek.brich@devl.cz>
parents: 0
diff changeset
    48
anymore thus being practically dead, while alternatives are not mature enough.
a9e12b7cc207 Fix MyManager. Add patch for MySQLdb and Python3.2. Add DelayedQuery.
Radek Brich <radek.brich@devl.cz>
parents: 0
diff changeset
    49
a9e12b7cc207 Fix MyManager. Add patch for MySQLdb and Python3.2. Add DelayedQuery.
Radek Brich <radek.brich@devl.cz>
parents: 0
diff changeset
    50
Just use PostgreSQL with Python, it's better option in most use cases.
a9e12b7cc207 Fix MyManager. Add patch for MySQLdb and Python3.2. Add DelayedQuery.
Radek Brich <radek.brich@devl.cz>
parents: 0
diff changeset
    51
34
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    52
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    53
Tools
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    54
=====
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    55
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    56
tablecopy.py
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    57
------------
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    58
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    59
./tablecapy.py db1 db2 -n
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    60
  Read all tables and all schema from db1, print table names. Remove -n to copy data to db2.
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    61
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    62
./tablecopy.py db1 db2 -s myschema
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    63
  Copy all tables in "myschema" from db1 to db2.
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    64
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    65
./tablecopy.py db1 db2 -s myschema --dst-schema otherschema
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    66
  Copy all tables in "myschema" from db1 to same tables in "otherschema" in db2.
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    67
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    68
./tablecopy.py db1 db2 -s ^my --regex
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    69
  Copy all tables from all schemas beginning with "my".
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    70
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    71
./tablecopy.py db1 db2 -s myschema1 --dst-schema myschema2 --regex -t ^my
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    72
  Copy all tables beginning with "my" from myschema1 to tables of same name in myschema2.
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    73
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    74
Rules:
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    75
  If nothing is specified, everything is copied.
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    76
  If no target schema or table is specified, data are copied to schema or table of same name.
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    77
  If --regex is specified, it applies to table name if specified, schema name otherwise.
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    78
  If both --regex and --src-table is specified, both source and dest schema can be named (no regex applied to them).
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    79
  Default schema name is '', which as regex matches all schemas. Same for table name.
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    80
  Regexes are match with source database, destination database is not checked (table may not exists, will fail when copying).
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    81
  Directly specified table name (no --regex) is not checked (may not exists).
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 18
diff changeset
    82