author | Radek Brich <brich.radek@ifortuna.cz> |
Tue, 06 May 2014 18:37:43 +0200 | |
changeset 101 | 2a2d0d5df03b |
parent 83 | 515fadd3d286 |
permissions | -rw-r--r-- |
6 | 1 |
# -*- coding: utf-8 -*- |
2 |
# |
|
3 |
# PgDataDiff - compare tables, print data differencies |
|
4 |
# |
|
5 |
# Copyright (c) 2011 Radek Brich <radek.brich@devl.cz> |
|
6 |
# |
|
7 |
# Permission is hereby granted, free of charge, to any person obtaining a copy |
|
8 |
# of this software and associated documentation files (the "Software"), to deal |
|
9 |
# in the Software without restriction, including without limitation the rights |
|
10 |
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell |
|
11 |
# copies of the Software, and to permit persons to whom the Software is |
|
12 |
# furnished to do so, subject to the following conditions: |
|
13 |
# |
|
14 |
# The above copyright notice and this permission notice shall be included in |
|
15 |
# all copies or substantial portions of the Software. |
|
16 |
# |
|
17 |
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR |
|
18 |
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, |
|
19 |
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE |
|
20 |
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER |
|
21 |
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, |
|
22 |
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN |
|
23 |
# THE SOFTWARE. |
|
24 |
||
25 |
||
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
26 |
from collections import OrderedDict |
6 | 27 |
|
9
2fcc8ef0b97d
Reorganize again :-) Add setup.py.
Radek Brich <radek.brich@devl.cz>
parents:
7
diff
changeset
|
28 |
from pgtoolkit import pgbrowser |
83
515fadd3d286
Add dependency on pycolib. Move common modules to pycolib. Add example table schema for meta DB.
Radek Brich <radek.brich@devl.cz>
parents:
54
diff
changeset
|
29 |
from pycolib.ansicolor import * |
6 | 30 |
|
101
2a2d0d5df03b
Refactor ToolBase to allow tool composition. Add TableSync tool (composited). Move more tools under pgtool.
Radek Brich <brich.radek@ifortuna.cz>
parents:
83
diff
changeset
|
31 |
import sys |
2a2d0d5df03b
Refactor ToolBase to allow tool composition. Add TableSync tool (composited). Move more tools under pgtool.
Radek Brich <brich.radek@ifortuna.cz>
parents:
83
diff
changeset
|
32 |
|
6 | 33 |
|
34 |
class DiffData: |
|
35 |
COLORS = { |
|
36 |
'+' : BOLD | GREEN, |
|
37 |
'-' : BOLD | RED, |
|
38 |
'*' : BOLD | YELLOW, |
|
27
5fb4883604d6
Add analyzeall tool. Updates, fixes.
Radek Brich <radek.brich@devl.cz>
parents:
14
diff
changeset
|
39 |
'V' : BOLD | WHITE, |
5fb4883604d6
Add analyzeall tool. Updates, fixes.
Radek Brich <radek.brich@devl.cz>
parents:
14
diff
changeset
|
40 |
'K' : BOLD | BLUE} |
54 | 41 |
|
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
42 |
def __init__(self, change, cols1, cols2, key=None): |
101
2a2d0d5df03b
Refactor ToolBase to allow tool composition. Add TableSync tool (composited). Move more tools under pgtool.
Radek Brich <brich.radek@ifortuna.cz>
parents:
83
diff
changeset
|
43 |
""" |
54 | 44 |
|
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
45 |
change - one of '+', '-', '*' (add, remove, update) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
46 |
cols1 - original column values (OrderedDict) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
47 |
cols2 - new column values (OrderedDict) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
48 |
key - primary key columns (OrderedDict) |
54 | 49 |
|
101
2a2d0d5df03b
Refactor ToolBase to allow tool composition. Add TableSync tool (composited). Move more tools under pgtool.
Radek Brich <brich.radek@ifortuna.cz>
parents:
83
diff
changeset
|
50 |
""" |
6 | 51 |
self.change = change |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
52 |
self.cols1 = cols1 |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
53 |
self.cols2 = cols2 |
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
54 |
self.key = key |
54 | 55 |
|
6 | 56 |
def format(self): |
57 |
out = [] |
|
54 | 58 |
|
6 | 59 |
out.append(highlight(1, self.COLORS[self.change])) |
60 |
out.extend([self.change, ' ']) |
|
54 | 61 |
|
6 | 62 |
out.extend(self._format_changes()) |
54 | 63 |
|
6 | 64 |
out.append(highlight(0)) |
54 | 65 |
|
6 | 66 |
return ''.join(out) |
67 |
||
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
68 |
def format_patch(self, table): |
6 | 69 |
method = { |
70 |
'+' : self._format_insert, |
|
71 |
'-' : self._format_delete, |
|
72 |
'*' : self._format_update} |
|
54 | 73 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
74 |
return method[self.change](table) |
6 | 75 |
|
54 | 76 |
def _format_changes(self): |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
77 |
if self.cols1 and not self.cols2: |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
78 |
return [', '.join([self._format_value_del(*x) for x in self.cols1.items()])] |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
79 |
if not self.cols1 and self.cols2: |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
80 |
return [', '.join([self._format_value_add(*x) for x in self.cols2.items()])] |
54 | 81 |
|
82 |
out = [] |
|
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
83 |
if self.key: |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
84 |
for colname in self.key: |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
85 |
out.extend([highlight(1, self.COLORS['*']), colname, ': ', highlight(0), self.key[colname], ', ']) |
27
5fb4883604d6
Add analyzeall tool. Updates, fixes.
Radek Brich <radek.brich@devl.cz>
parents:
14
diff
changeset
|
86 |
|
6 | 87 |
items = [] |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
88 |
for i in range(len(self.cols1)): |
6 | 89 |
items.append(( |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
90 |
list(self.cols1.keys())[i], |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
91 |
list(self.cols1.values())[i], |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
92 |
list(self.cols2.values())[i])) |
27
5fb4883604d6
Add analyzeall tool. Updates, fixes.
Radek Brich <radek.brich@devl.cz>
parents:
14
diff
changeset
|
93 |
out.extend([', '.join([self._format_value_change(*x) for x in items])]) |
54 | 94 |
|
27
5fb4883604d6
Add analyzeall tool. Updates, fixes.
Radek Brich <radek.brich@devl.cz>
parents:
14
diff
changeset
|
95 |
return out |
6 | 96 |
|
97 |
def _format_value_del(self, k, v): |
|
98 |
fs = (highlight(1, self.COLORS['-']) + '{}: ' + highlight(0) + '{}') |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
99 |
return fs.format(k, v) |
6 | 100 |
|
101 |
def _format_value_add(self, k, v): |
|
54 | 102 |
fs = (highlight(1, self.COLORS['+']) + '{}: ' + highlight(0) + |
6 | 103 |
highlight(1, self.COLORS['V']) + '{}' + highlight(0)) |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
104 |
return fs.format(k, v) |
6 | 105 |
|
106 |
def _format_value_change(self, k, v1, v2): |
|
54 | 107 |
fs = (highlight(1, self.COLORS['*']) + '{}: ' + highlight(0) + |
6 | 108 |
'{} â–¶ ' + |
109 |
highlight(1, self.COLORS['V']) + '{}' + highlight(0)) |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
110 |
return fs.format(k, v1, v2) |
6 | 111 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
112 |
def _format_insert(self, table): |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
113 |
out = ['INSERT INTO ', table, ' ('] |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
114 |
out.append(', '.join(self.cols2.keys())) |
6 | 115 |
out.append(') VALUES (') |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
116 |
out.append(', '.join(self.cols2.values())) |
6 | 117 |
out.append(');') |
118 |
return ''.join(out) |
|
54 | 119 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
120 |
def _format_delete(self, table): |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
121 |
out = ['DELETE FROM ', table] |
54 | 122 |
out.extend(self._format_where()) |
6 | 123 |
return ''.join(out) |
54 | 124 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
125 |
def _format_update(self, table): |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
126 |
out = ['UPDATE ', table, ' SET '] |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
127 |
out.append(', '.join([self._format_set(*x) for x in self.cols2.items()])) |
6 | 128 |
out.extend(self._format_where()) |
129 |
return ''.join(out) |
|
130 |
||
131 |
def _format_set(self, k, v): |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
132 |
return '{} = {}'.format(k, v) |
6 | 133 |
|
134 |
def _format_where(self): |
|
135 |
out = [' WHERE '] |
|
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
136 |
for colname in self.key: |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
137 |
out.extend([colname, ' = ', self.key[colname], ' AND ']) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
138 |
out[-1] = ';' |
6 | 139 |
return out |
140 |
||
141 |
class PgDataDiff: |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
142 |
def __init__(self, conn1, conn2): |
6 | 143 |
self.allowcolor = False |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
144 |
self.conn1 = conn1 |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
145 |
self.conn2 = conn2 |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
146 |
self.fulltable1 = None |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
147 |
self.fulltable2 = None |
54 | 148 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
149 |
def settable1(self, table, schema='public'): |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
150 |
self.schema1 = schema |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
151 |
self.table1 = table |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
152 |
self.fulltable1 = '"' + schema + '"."'+ table + '"' |
54 | 153 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
154 |
def settable2(self, table, schema='public'): |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
155 |
self.schema2 = schema |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
156 |
self.table2 = table |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
157 |
self.fulltable2 = '"' + schema + '"."'+ table + '"' |
54 | 158 |
|
6 | 159 |
def iter_diff(self): |
101
2a2d0d5df03b
Refactor ToolBase to allow tool composition. Add TableSync tool (composited). Move more tools under pgtool.
Radek Brich <brich.radek@ifortuna.cz>
parents:
83
diff
changeset
|
160 |
"""Return differencies between data of two tables. |
54 | 161 |
|
6 | 162 |
Yields one line at the time. |
54 | 163 |
|
101
2a2d0d5df03b
Refactor ToolBase to allow tool composition. Add TableSync tool (composited). Move more tools under pgtool.
Radek Brich <brich.radek@ifortuna.cz>
parents:
83
diff
changeset
|
164 |
""" |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
165 |
curs1, curs2 = self._select() |
54 | 166 |
|
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
167 |
row1 = curs1.fetchone_dict() |
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
168 |
row2 = curs2.fetchone_dict() |
54 | 169 |
|
6 | 170 |
while True: |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
171 |
if row1 is None and row2 is None: |
6 | 172 |
break |
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
173 |
diff = self._compare_row(row1, row2, curs1.adapt, curs2.adapt) |
54 | 174 |
|
6 | 175 |
if diff: |
176 |
yield diff |
|
54 | 177 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
178 |
if diff.change == '-': |
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
179 |
row1 = curs1.fetchone_dict() |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
180 |
continue |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
181 |
if diff.change == '+': |
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
182 |
row2 = curs2.fetchone_dict() |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
183 |
continue |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
184 |
# change == '*' or not diff |
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
185 |
row1 = curs1.fetchone_dict() |
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
186 |
row2 = curs2.fetchone_dict() |
54 | 187 |
|
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
188 |
curs1.close() |
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
189 |
curs2.close() |
54 | 190 |
|
101
2a2d0d5df03b
Refactor ToolBase to allow tool composition. Add TableSync tool (composited). Move more tools under pgtool.
Radek Brich <brich.radek@ifortuna.cz>
parents:
83
diff
changeset
|
191 |
def print_diff(self, file=sys.stdout): |
2a2d0d5df03b
Refactor ToolBase to allow tool composition. Add TableSync tool (composited). Move more tools under pgtool.
Radek Brich <brich.radek@ifortuna.cz>
parents:
83
diff
changeset
|
192 |
"""Print differencies between data of two tables. |
54 | 193 |
|
6 | 194 |
The output is in human readable form. |
54 | 195 |
|
6 | 196 |
Set allowcolor=True of PgDataDiff instance to get colored output. |
54 | 197 |
|
101
2a2d0d5df03b
Refactor ToolBase to allow tool composition. Add TableSync tool (composited). Move more tools under pgtool.
Radek Brich <brich.radek@ifortuna.cz>
parents:
83
diff
changeset
|
198 |
""" |
6 | 199 |
for ln in self.iter_diff(): |
101
2a2d0d5df03b
Refactor ToolBase to allow tool composition. Add TableSync tool (composited). Move more tools under pgtool.
Radek Brich <brich.radek@ifortuna.cz>
parents:
83
diff
changeset
|
200 |
print(ln.format(), file=file) |
54 | 201 |
|
101
2a2d0d5df03b
Refactor ToolBase to allow tool composition. Add TableSync tool (composited). Move more tools under pgtool.
Radek Brich <brich.radek@ifortuna.cz>
parents:
83
diff
changeset
|
202 |
def print_patch(self, file=sys.stdout): |
2a2d0d5df03b
Refactor ToolBase to allow tool composition. Add TableSync tool (composited). Move more tools under pgtool.
Radek Brich <brich.radek@ifortuna.cz>
parents:
83
diff
changeset
|
203 |
"""Print SQL script usable as patch for destination table. |
54 | 204 |
|
6 | 205 |
Supports INSERT, DELETE and UPDATE operations. |
54 | 206 |
|
101
2a2d0d5df03b
Refactor ToolBase to allow tool composition. Add TableSync tool (composited). Move more tools under pgtool.
Radek Brich <brich.radek@ifortuna.cz>
parents:
83
diff
changeset
|
207 |
""" |
6 | 208 |
for ln in self.iter_diff(): |
101
2a2d0d5df03b
Refactor ToolBase to allow tool composition. Add TableSync tool (composited). Move more tools under pgtool.
Radek Brich <brich.radek@ifortuna.cz>
parents:
83
diff
changeset
|
209 |
print(ln.format_patch(self.fulltable1), file=file) |
6 | 210 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
211 |
def _select(self): |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
212 |
browser = pgbrowser.PgBrowser(self.conn1) |
54 | 213 |
|
12 | 214 |
columns = browser.list_columns(schema=self.schema1, table=self.table1, order=1) |
27
5fb4883604d6
Add analyzeall tool. Updates, fixes.
Radek Brich <radek.brich@devl.cz>
parents:
14
diff
changeset
|
215 |
if not columns: |
5fb4883604d6
Add analyzeall tool. Updates, fixes.
Radek Brich <radek.brich@devl.cz>
parents:
14
diff
changeset
|
216 |
raise Exception('Table %s.%s not found.' % (self.schema1, self.table1)) |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
217 |
columns_sel = ', '.join(['"' + x['name'] + '"' for x in columns]) |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
218 |
self.colnames = [x['name'] for x in columns] |
54 | 219 |
|
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
220 |
pkey = [ind for ind in browser.list_indexes(schema=self.schema1, table=self.table1) if ind['primary']] |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
221 |
if not pkey: |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
222 |
raise Exception('Table %s.%s has no primary key.' % (self.schema1, self.table1)) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
223 |
pkey = pkey[0] |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
224 |
pkey_sel = ', '.join(['"' + x + '"' for x in pkey['columns']]) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
225 |
self.pkeycolnames = pkey['columns'] |
54 | 226 |
|
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
227 |
query1 = 'SELECT ' + columns_sel + ' FROM ' + self.fulltable1 + ' ORDER BY ' + pkey_sel |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
228 |
query2 = 'SELECT ' + columns_sel + ' FROM ' + self.fulltable2 + ' ORDER BY ' + pkey_sel |
54 | 229 |
|
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
230 |
curs1 = self.conn1.cursor('curs1') |
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
231 |
curs2 = self.conn2.cursor('curs2') |
54 | 232 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
233 |
curs1.execute(query1) |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
234 |
curs2.execute(query2) |
54 | 235 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
236 |
return curs1, curs2 |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
237 |
|
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
238 |
def _compare_data(self, row1, row2): |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
239 |
cols1 = OrderedDict() |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
240 |
cols2 = OrderedDict() |
54 | 241 |
for name in row1.keys(): |
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
242 |
if row1[name] != row2[name]: |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
243 |
cols1[name] = row1[name] |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
244 |
cols2[name] = row2[name] |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
245 |
if cols1: |
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
246 |
key = OrderedDict(zip(self.pkeycolnames, [row1[colname] for colname in self.pkeycolnames])) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
247 |
return DiffData('*', cols1, cols2, key=key) |
54 | 248 |
|
6 | 249 |
return None |
54 | 250 |
|
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
251 |
def _compare_row(self, row1, row2, adapt1, adapt2): |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
252 |
if row2 is None: |
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
253 |
row1 = adapt1(row1) |
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
254 |
key = OrderedDict(zip(self.pkeycolnames, [row1[colname] for colname in self.pkeycolnames])) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
255 |
return DiffData('-', row1, None, key=key) |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
256 |
if row1 is None: |
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
257 |
row2 = adapt2(row2) |
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
258 |
return DiffData('+', None, row2) |
54 | 259 |
|
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
260 |
for keyname in self.pkeycolnames: |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
261 |
if row1[keyname] < row2[keyname]: |
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
262 |
row1 = adapt1(row1) |
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
263 |
key = OrderedDict(zip(self.pkeycolnames, [row1[colname] for colname in self.pkeycolnames])) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
264 |
return DiffData('-', row1, None, key=key) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
265 |
for keyname in self.pkeycolnames: |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
266 |
if row1[keyname] > row2[keyname]: |
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
267 |
row2 = adapt2(row2) |
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
268 |
return DiffData('+', None, row2) |
54 | 269 |
|
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
270 |
row1 = adapt1(row1) |
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
271 |
row2 = adapt2(row2) |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
272 |
return self._compare_data(row1, row2) |
6 | 273 |