author | Radek Brich <brich.radek@ifortuna.cz> |
Tue, 06 May 2014 18:34:38 +0200 | |
changeset 99 | 245646538743 |
parent 83 | 515fadd3d286 |
child 101 | 2a2d0d5df03b |
permissions | -rw-r--r-- |
6 | 1 |
# -*- coding: utf-8 -*- |
2 |
# |
|
3 |
# PgDataDiff - compare tables, print data differencies |
|
4 |
# |
|
5 |
# Copyright (c) 2011 Radek Brich <radek.brich@devl.cz> |
|
6 |
# |
|
7 |
# Permission is hereby granted, free of charge, to any person obtaining a copy |
|
8 |
# of this software and associated documentation files (the "Software"), to deal |
|
9 |
# in the Software without restriction, including without limitation the rights |
|
10 |
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell |
|
11 |
# copies of the Software, and to permit persons to whom the Software is |
|
12 |
# furnished to do so, subject to the following conditions: |
|
13 |
# |
|
14 |
# The above copyright notice and this permission notice shall be included in |
|
15 |
# all copies or substantial portions of the Software. |
|
16 |
# |
|
17 |
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR |
|
18 |
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, |
|
19 |
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE |
|
20 |
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER |
|
21 |
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, |
|
22 |
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN |
|
23 |
# THE SOFTWARE. |
|
24 |
||
25 |
||
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
26 |
from collections import OrderedDict |
6 | 27 |
|
9
2fcc8ef0b97d
Reorganize again :-) Add setup.py.
Radek Brich <radek.brich@devl.cz>
parents:
7
diff
changeset
|
28 |
from pgtoolkit import pgbrowser |
83
515fadd3d286
Add dependency on pycolib. Move common modules to pycolib. Add example table schema for meta DB.
Radek Brich <radek.brich@devl.cz>
parents:
54
diff
changeset
|
29 |
from pycolib.ansicolor import * |
6 | 30 |
|
31 |
||
32 |
class DiffData: |
|
33 |
COLORS = { |
|
34 |
'+' : BOLD | GREEN, |
|
35 |
'-' : BOLD | RED, |
|
36 |
'*' : BOLD | YELLOW, |
|
27
5fb4883604d6
Add analyzeall tool. Updates, fixes.
Radek Brich <radek.brich@devl.cz>
parents:
14
diff
changeset
|
37 |
'V' : BOLD | WHITE, |
5fb4883604d6
Add analyzeall tool. Updates, fixes.
Radek Brich <radek.brich@devl.cz>
parents:
14
diff
changeset
|
38 |
'K' : BOLD | BLUE} |
54 | 39 |
|
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
40 |
def __init__(self, change, cols1, cols2, key=None): |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
41 |
''' |
54 | 42 |
|
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
43 |
change - one of '+', '-', '*' (add, remove, update) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
44 |
cols1 - original column values (OrderedDict) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
45 |
cols2 - new column values (OrderedDict) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
46 |
key - primary key columns (OrderedDict) |
54 | 47 |
|
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
48 |
''' |
6 | 49 |
self.change = change |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
50 |
self.cols1 = cols1 |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
51 |
self.cols2 = cols2 |
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
52 |
self.key = key |
54 | 53 |
|
6 | 54 |
def format(self): |
55 |
out = [] |
|
54 | 56 |
|
6 | 57 |
out.append(highlight(1, self.COLORS[self.change])) |
58 |
out.extend([self.change, ' ']) |
|
54 | 59 |
|
6 | 60 |
out.extend(self._format_changes()) |
54 | 61 |
|
6 | 62 |
out.append(highlight(0)) |
54 | 63 |
|
6 | 64 |
return ''.join(out) |
65 |
||
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
66 |
def format_patch(self, table): |
6 | 67 |
method = { |
68 |
'+' : self._format_insert, |
|
69 |
'-' : self._format_delete, |
|
70 |
'*' : self._format_update} |
|
54 | 71 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
72 |
return method[self.change](table) |
6 | 73 |
|
54 | 74 |
def _format_changes(self): |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
75 |
if self.cols1 and not self.cols2: |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
76 |
return [', '.join([self._format_value_del(*x) for x in self.cols1.items()])] |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
77 |
if not self.cols1 and self.cols2: |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
78 |
return [', '.join([self._format_value_add(*x) for x in self.cols2.items()])] |
54 | 79 |
|
80 |
out = [] |
|
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
81 |
if self.key: |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
82 |
for colname in self.key: |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
83 |
out.extend([highlight(1, self.COLORS['*']), colname, ': ', highlight(0), self.key[colname], ', ']) |
27
5fb4883604d6
Add analyzeall tool. Updates, fixes.
Radek Brich <radek.brich@devl.cz>
parents:
14
diff
changeset
|
84 |
|
6 | 85 |
items = [] |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
86 |
for i in range(len(self.cols1)): |
6 | 87 |
items.append(( |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
88 |
list(self.cols1.keys())[i], |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
89 |
list(self.cols1.values())[i], |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
90 |
list(self.cols2.values())[i])) |
27
5fb4883604d6
Add analyzeall tool. Updates, fixes.
Radek Brich <radek.brich@devl.cz>
parents:
14
diff
changeset
|
91 |
out.extend([', '.join([self._format_value_change(*x) for x in items])]) |
54 | 92 |
|
27
5fb4883604d6
Add analyzeall tool. Updates, fixes.
Radek Brich <radek.brich@devl.cz>
parents:
14
diff
changeset
|
93 |
return out |
6 | 94 |
|
95 |
def _format_value_del(self, k, v): |
|
96 |
fs = (highlight(1, self.COLORS['-']) + '{}: ' + highlight(0) + '{}') |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
97 |
return fs.format(k, v) |
6 | 98 |
|
99 |
def _format_value_add(self, k, v): |
|
54 | 100 |
fs = (highlight(1, self.COLORS['+']) + '{}: ' + highlight(0) + |
6 | 101 |
highlight(1, self.COLORS['V']) + '{}' + highlight(0)) |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
102 |
return fs.format(k, v) |
6 | 103 |
|
104 |
def _format_value_change(self, k, v1, v2): |
|
54 | 105 |
fs = (highlight(1, self.COLORS['*']) + '{}: ' + highlight(0) + |
6 | 106 |
'{} â–¶ ' + |
107 |
highlight(1, self.COLORS['V']) + '{}' + highlight(0)) |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
108 |
return fs.format(k, v1, v2) |
6 | 109 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
110 |
def _format_insert(self, table): |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
111 |
out = ['INSERT INTO ', table, ' ('] |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
112 |
out.append(', '.join(self.cols2.keys())) |
6 | 113 |
out.append(') VALUES (') |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
114 |
out.append(', '.join(self.cols2.values())) |
6 | 115 |
out.append(');') |
116 |
return ''.join(out) |
|
54 | 117 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
118 |
def _format_delete(self, table): |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
119 |
out = ['DELETE FROM ', table] |
54 | 120 |
out.extend(self._format_where()) |
6 | 121 |
return ''.join(out) |
54 | 122 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
123 |
def _format_update(self, table): |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
124 |
out = ['UPDATE ', table, ' SET '] |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
125 |
out.append(', '.join([self._format_set(*x) for x in self.cols2.items()])) |
6 | 126 |
out.extend(self._format_where()) |
127 |
return ''.join(out) |
|
128 |
||
129 |
def _format_set(self, k, v): |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
130 |
return '{} = {}'.format(k, v) |
6 | 131 |
|
132 |
def _format_where(self): |
|
133 |
out = [' WHERE '] |
|
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
134 |
for colname in self.key: |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
135 |
out.extend([colname, ' = ', self.key[colname], ' AND ']) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
136 |
out[-1] = ';' |
6 | 137 |
return out |
138 |
||
139 |
class PgDataDiff: |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
140 |
def __init__(self, conn1, conn2): |
6 | 141 |
self.allowcolor = False |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
142 |
self.conn1 = conn1 |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
143 |
self.conn2 = conn2 |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
144 |
self.fulltable1 = None |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
145 |
self.fulltable2 = None |
54 | 146 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
147 |
def settable1(self, table, schema='public'): |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
148 |
self.schema1 = schema |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
149 |
self.table1 = table |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
150 |
self.fulltable1 = '"' + schema + '"."'+ table + '"' |
54 | 151 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
152 |
def settable2(self, table, schema='public'): |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
153 |
self.schema2 = schema |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
154 |
self.table2 = table |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
155 |
self.fulltable2 = '"' + schema + '"."'+ table + '"' |
54 | 156 |
|
6 | 157 |
def iter_diff(self): |
158 |
'''Return differencies between data of two tables. |
|
54 | 159 |
|
6 | 160 |
Yields one line at the time. |
54 | 161 |
|
6 | 162 |
''' |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
163 |
curs1, curs2 = self._select() |
54 | 164 |
|
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
165 |
row1 = curs1.fetchone_dict() |
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
166 |
row2 = curs2.fetchone_dict() |
54 | 167 |
|
6 | 168 |
while True: |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
169 |
if row1 is None and row2 is None: |
6 | 170 |
break |
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
171 |
diff = self._compare_row(row1, row2, curs1.adapt, curs2.adapt) |
54 | 172 |
|
6 | 173 |
if diff: |
174 |
yield diff |
|
54 | 175 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
176 |
if diff.change == '-': |
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
177 |
row1 = curs1.fetchone_dict() |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
178 |
continue |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
179 |
if diff.change == '+': |
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
180 |
row2 = curs2.fetchone_dict() |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
181 |
continue |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
182 |
# change == '*' or not diff |
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
183 |
row1 = curs1.fetchone_dict() |
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
184 |
row2 = curs2.fetchone_dict() |
54 | 185 |
|
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
186 |
curs1.close() |
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
187 |
curs2.close() |
54 | 188 |
|
6 | 189 |
def print_diff(self): |
190 |
'''Print differencies between data of two tables. |
|
54 | 191 |
|
6 | 192 |
The output is in human readable form. |
54 | 193 |
|
6 | 194 |
Set allowcolor=True of PgDataDiff instance to get colored output. |
54 | 195 |
|
6 | 196 |
''' |
197 |
for ln in self.iter_diff(): |
|
198 |
print(ln.format()) |
|
54 | 199 |
|
6 | 200 |
def print_patch(self): |
201 |
'''Print SQL script usable as patch for destination table. |
|
54 | 202 |
|
6 | 203 |
Supports INSERT, DELETE and UPDATE operations. |
54 | 204 |
|
6 | 205 |
''' |
206 |
for ln in self.iter_diff(): |
|
14
a900bc629ecc
TableDiffTool: add arguments to set different destination schema and table. PgDataDiff: fixes.
Radek Brich <radek.brich@devl.cz>
parents:
12
diff
changeset
|
207 |
print(ln.format_patch(self.fulltable1)) |
6 | 208 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
209 |
def _select(self): |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
210 |
browser = pgbrowser.PgBrowser(self.conn1) |
54 | 211 |
|
12 | 212 |
columns = browser.list_columns(schema=self.schema1, table=self.table1, order=1) |
27
5fb4883604d6
Add analyzeall tool. Updates, fixes.
Radek Brich <radek.brich@devl.cz>
parents:
14
diff
changeset
|
213 |
if not columns: |
5fb4883604d6
Add analyzeall tool. Updates, fixes.
Radek Brich <radek.brich@devl.cz>
parents:
14
diff
changeset
|
214 |
raise Exception('Table %s.%s not found.' % (self.schema1, self.table1)) |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
215 |
columns_sel = ', '.join(['"' + x['name'] + '"' for x in columns]) |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
216 |
self.colnames = [x['name'] for x in columns] |
54 | 217 |
|
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
218 |
pkey = [ind for ind in browser.list_indexes(schema=self.schema1, table=self.table1) if ind['primary']] |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
219 |
if not pkey: |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
220 |
raise Exception('Table %s.%s has no primary key.' % (self.schema1, self.table1)) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
221 |
pkey = pkey[0] |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
222 |
pkey_sel = ', '.join(['"' + x + '"' for x in pkey['columns']]) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
223 |
self.pkeycolnames = pkey['columns'] |
54 | 224 |
|
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
225 |
query1 = 'SELECT ' + columns_sel + ' FROM ' + self.fulltable1 + ' ORDER BY ' + pkey_sel |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
226 |
query2 = 'SELECT ' + columns_sel + ' FROM ' + self.fulltable2 + ' ORDER BY ' + pkey_sel |
54 | 227 |
|
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
228 |
curs1 = self.conn1.cursor('curs1') |
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
229 |
curs2 = self.conn2.cursor('curs2') |
54 | 230 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
231 |
curs1.execute(query1) |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
232 |
curs2.execute(query2) |
54 | 233 |
|
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
234 |
return curs1, curs2 |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
235 |
|
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
236 |
def _compare_data(self, row1, row2): |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
237 |
cols1 = OrderedDict() |
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
238 |
cols2 = OrderedDict() |
54 | 239 |
for name in row1.keys(): |
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
240 |
if row1[name] != row2[name]: |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
241 |
cols1[name] = row1[name] |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
242 |
cols2[name] = row2[name] |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
243 |
if cols1: |
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
244 |
key = OrderedDict(zip(self.pkeycolnames, [row1[colname] for colname in self.pkeycolnames])) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
245 |
return DiffData('*', cols1, cols2, key=key) |
54 | 246 |
|
6 | 247 |
return None |
54 | 248 |
|
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
249 |
def _compare_row(self, row1, row2, adapt1, adapt2): |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
250 |
if row2 is None: |
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
251 |
row1 = adapt1(row1) |
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
252 |
key = OrderedDict(zip(self.pkeycolnames, [row1[colname] for colname in self.pkeycolnames])) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
253 |
return DiffData('-', row1, None, key=key) |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
254 |
if row1 is None: |
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
255 |
row2 = adapt2(row2) |
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
256 |
return DiffData('+', None, row2) |
54 | 257 |
|
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
258 |
for keyname in self.pkeycolnames: |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
259 |
if row1[keyname] < row2[keyname]: |
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
260 |
row1 = adapt1(row1) |
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
261 |
key = OrderedDict(zip(self.pkeycolnames, [row1[colname] for colname in self.pkeycolnames])) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
262 |
return DiffData('-', row1, None, key=key) |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
263 |
for keyname in self.pkeycolnames: |
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
264 |
if row1[keyname] > row2[keyname]: |
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
265 |
row2 = adapt2(row2) |
31
c2e6e24b83d9
Add browser - database schema browser using tuikit (curses UI). Add listdepends - tool which shows depending views for column. Update pgdatadiff - allow composite primary key. Update pgmanager - RowDict is now OrderedDict. Drop support for Python2.x.
Radek Brich <radek.brich@devl.cz>
parents:
27
diff
changeset
|
266 |
return DiffData('+', None, row2) |
54 | 267 |
|
41
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
268 |
row1 = adapt1(row1) |
6aad5e35efe8
PgDataDiff: Fix sorting - do not adapt primary key before sort condition.
Radek Brich <radek.brich@devl.cz>
parents:
31
diff
changeset
|
269 |
row2 = adapt2(row2) |
7
685b20d2d3ab
Reorganize directories. PgDataDiff - reworked. PgManager - add fetchone_adapted, fetchall_adapted to cursor.
Radek Brich <radek.brich@devl.cz>
parents:
6
diff
changeset
|
270 |
return self._compare_data(row1, row2) |
6 | 271 |