pgtoolkit/tools/tablediff.py
author Radek Brich <brich.radek@ifortuna.cz>
Tue, 06 May 2014 18:37:41 +0200
changeset 100 d6088dba8fea
parent 93 tablediff.py@b72591087495
child 101 2a2d0d5df03b
permissions -rw-r--r--
Add pgtool wrapper for all tools. Only this script will be installed into system bin.
Ignore whitespace changes - Everywhere: Within whitespace: At end of lines:
34
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
     1
from pgtoolkit import toolbase, pgmanager, pgdatadiff
100
d6088dba8fea Add pgtool wrapper for all tools. Only this script will be installed into system bin.
Radek Brich <brich.radek@ifortuna.cz>
parents: 93
diff changeset
     2
from pgtoolkit.toolbase import SrcDstTablesTool
83
515fadd3d286 Add dependency on pycolib. Move common modules to pycolib. Add example table schema for meta DB.
Radek Brich <radek.brich@devl.cz>
parents: 56
diff changeset
     3
from pycolib.ansicolor import highlight, BOLD, YELLOW
6
4ab077c93b2d Add table diff tool.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
     4
4ab077c93b2d Add table diff tool.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
     5
34
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
     6
class TableDiffTool(toolbase.SrcDstTablesTool):
100
d6088dba8fea Add pgtool wrapper for all tools. Only this script will be installed into system bin.
Radek Brich <brich.radek@ifortuna.cz>
parents: 93
diff changeset
     7
d6088dba8fea Add pgtool wrapper for all tools. Only this script will be installed into system bin.
Radek Brich <brich.radek@ifortuna.cz>
parents: 93
diff changeset
     8
    """
d6088dba8fea Add pgtool wrapper for all tools. Only this script will be installed into system bin.
Radek Brich <brich.radek@ifortuna.cz>
parents: 93
diff changeset
     9
    Print differencies between data in tables.
d6088dba8fea Add pgtool wrapper for all tools. Only this script will be installed into system bin.
Radek Brich <brich.radek@ifortuna.cz>
parents: 93
diff changeset
    10
d6088dba8fea Add pgtool wrapper for all tools. Only this script will be installed into system bin.
Radek Brich <brich.radek@ifortuna.cz>
parents: 93
diff changeset
    11
    Requirements:
d6088dba8fea Add pgtool wrapper for all tools. Only this script will be installed into system bin.
Radek Brich <brich.radek@ifortuna.cz>
parents: 93
diff changeset
    12
    * Source table must have defined PRIMARY KEY.
d6088dba8fea Add pgtool wrapper for all tools. Only this script will be installed into system bin.
Radek Brich <brich.radek@ifortuna.cz>
parents: 93
diff changeset
    13
    * Destination table must contain all columns from source table.
d6088dba8fea Add pgtool wrapper for all tools. Only this script will be installed into system bin.
Radek Brich <brich.radek@ifortuna.cz>
parents: 93
diff changeset
    14
      Order is not important.
d6088dba8fea Add pgtool wrapper for all tools. Only this script will be installed into system bin.
Radek Brich <brich.radek@ifortuna.cz>
parents: 93
diff changeset
    15
d6088dba8fea Add pgtool wrapper for all tools. Only this script will be installed into system bin.
Radek Brich <brich.radek@ifortuna.cz>
parents: 93
diff changeset
    16
    """
d6088dba8fea Add pgtool wrapper for all tools. Only this script will be installed into system bin.
Radek Brich <brich.radek@ifortuna.cz>
parents: 93
diff changeset
    17
6
4ab077c93b2d Add table diff tool.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
    18
    def __init__(self):
100
d6088dba8fea Add pgtool wrapper for all tools. Only this script will be installed into system bin.
Radek Brich <brich.radek@ifortuna.cz>
parents: 93
diff changeset
    19
        SrcDstTablesTool.__init__(self, name='tablediff', desc=self.__doc__, allow_reverse=True)
7
685b20d2d3ab Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents: 6
diff changeset
    20
        self.parser.add_argument('--sql', action='store_true', help='Output is SQL script.')
34
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    21
        self.parser.add_argument('--rowcount', action='store_true', help='Compare number of rows.')
56
94e091c23ebb Add reverse parameter for diff tools.
Radek Brich <radek.brich@devl.cz>
parents: 41
diff changeset
    22
6
4ab077c93b2d Add table diff tool.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
    23
    def main(self):
34
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    24
        srcconn = self.pgm.get_conn('src')
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    25
        dstconn = self.pgm.get_conn('dst')
56
94e091c23ebb Add reverse parameter for diff tools.
Radek Brich <radek.brich@devl.cz>
parents: 41
diff changeset
    26
34
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    27
        dd = pgdatadiff.PgDataDiff(srcconn, dstconn)
56
94e091c23ebb Add reverse parameter for diff tools.
Radek Brich <radek.brich@devl.cz>
parents: 41
diff changeset
    28
34
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    29
        for srcschema, srctable, dstschema, dsttable in self.tables():
35
e7f79c4a27ce TableCopyTool: order tables with respect to references.
Radek Brich <radek.brich@devl.cz>
parents: 34
diff changeset
    30
            print('-- Diff from [%s] %s.%s to [%s] %s.%s' % (
34
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    31
                self.args.src, srcschema, srctable,
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    32
                self.args.dst, dstschema, dsttable))
56
94e091c23ebb Add reverse parameter for diff tools.
Radek Brich <radek.brich@devl.cz>
parents: 41
diff changeset
    33
34
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    34
            if self.args.rowcount:
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    35
                with self.pgm.cursor('src') as curs:
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    36
                    curs.execute('''SELECT count(*) FROM "%s"."%s"''' % (srcschema, srctable))
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    37
                    srccount = curs.fetchone()[0]
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    38
                with self.pgm.cursor('dst') as curs:
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    39
                    curs.execute('''SELECT count(*) FROM "%s"."%s"''' % (dstschema, dsttable))
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    40
                    dstcount = curs.fetchone()[0]
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    41
                if srccount != dstcount:
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    42
                    print(highlight(1, BOLD | YELLOW), "Row count differs: src=%s dst=%s" % (srccount, dstcount), highlight(0), sep='')
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    43
                continue
56
94e091c23ebb Add reverse parameter for diff tools.
Radek Brich <radek.brich@devl.cz>
parents: 41
diff changeset
    44
34
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    45
            dd.settable1(srctable, srcschema)
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    46
            dd.settable2(dsttable, dstschema)
56
94e091c23ebb Add reverse parameter for diff tools.
Radek Brich <radek.brich@devl.cz>
parents: 41
diff changeset
    47
34
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    48
            if self.args.sql:
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    49
                dd.print_patch()
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    50
            else:
98c7809af415 Add PgDataCopy. Add TableCopyTool.Add SrcDstTablesTool class to toolbase, use in tablecopy, tablediff.
Radek Brich <radek.brich@devl.cz>
parents: 14
diff changeset
    51
                dd.print_diff()
6
4ab077c93b2d Add table diff tool.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
    52
4ab077c93b2d Add table diff tool.
Radek Brich <radek.brich@devl.cz>
parents:
diff changeset
    53
100
d6088dba8fea Add pgtool wrapper for all tools. Only this script will be installed into system bin.
Radek Brich <brich.radek@ifortuna.cz>
parents: 93
diff changeset
    54
cls = TableDiffTool
7
685b20d2d3ab Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents: 6
diff changeset
    55