Skip to content

Latest commit

 

History

History
230 lines (172 loc) · 7.34 KB

README.md

File metadata and controls

230 lines (172 loc) · 7.34 KB
Raw

comparesv

Python CSV Comparison on steriods

Installation

pip install comparesv

Usage

comparesv [-h] [-v] [--enc1 ENCODING] [--enc2 ENCODING] [-i]
              [-rm ROW_MATCH] [-cm COLUMN_MATCH] [-sm STRING_MATCH] [-ir]
              [-ic] [-is] [-s]
              [FILE1] [FILE2]

CSV files comparison

positional arguments:
  FILE1                 the first CSV file
  FILE2                 the second CSV file

optional arguments:
  -h, --help            show this help message and exit
  -v, --version         show program's version number and exit
  --enc1 ENCODING       encoding of the first file (default is to autodetect)
  --enc2 ENCODING       encoding of the second file (default is to autodetect)
  -i, --ignore-case     ignore case (default is case-sensitive)
  -rm ROW_MATCH, --row-match ROW_MATCH
                        Logic to be used to identify the rows. Possible
                        options 'order', 'fuzzy', 'deep' (default is order)
  -cm COLUMN_MATCH, --column-match COLUMN_MATCH
                        Logic to be used to identify the columns. Possible
                        options 'exact','fuzzy' (default is exact)
  -sm STRING_MATCH, --string-match STRING_MATCH
                        Logic to be used to identify the columns. Possible
                        options 'exact','fuzzy' (default is exact)
  -ir, --include-addnl-rows
                        Include added additional added rows from second file
                        (default is false)
  -ic, --include-addnl-columns
                        Include added additional columns from second file
                        (default is false)
  -is, --include-stats  Include stats (default is false)
  -s, --save-output     Save output to file

Examples

Scenario 1: Simple direct comparison

id first last age
432 Roy Aguilar 46
914 Janie Bowman 24
021 Grace Copeland 53
708 Louise Franklin 25
850 Gertrude Carr 60

vs

id first last age
432 Roy Aguilar 46
914 Janie Bowman 24
021 Grace Copeland 53
708 Louise Franklin 25
850 Gertrude Carr 60 
comparesv file1 file2

Will provide:

S.No id first last age
1 True True True True
2 True True True True
3 True True True True
4 True True True True
5 True True True True

and

S.No id first last age
1 [432]:[432] [Roy]:[Roy] [Aguilar]:[Aguilar] [46]:[46]
2 [914]:[914] [Janie]:[Janie] [Bowman]:[Bowman] [24]:[24]
3 [021]:[021] [Grace]:[Grace] [Copeland]:[Copeland] [53]:[53]
4 [708]:[708] [Louise]:[Louise] [Franklin]:[Franklin] [25]:[25]
5 [850]:[850] [Gertrude]:[Gertrude] [Carr]:[Carr] [60]:[60]

Scenario 2: Fuzzy column names

id first last age of student
432 Roy Aguilar 46
914 Janie Bowman 24

and

id first last age
432 Roy Aguilar 46
914 Janie Bowman 24 
comparesv file1.csv file2.csv --column-match 'fuzzy'

will provide

S.No id first last age
1 True True True True
2 True True True True

Scenario 3: Fuzzy row order - Differnt ordered textual data

id first last age
432 Roy Aguilar 46
914 Janie Bowman 24
021 Grace Copeland 53

and

id first last age of student
021 Grace Copeland 53
432 Roy Aguilar 46
914 Janie Bowman 24 
comparesv file1.csv file2.csv --column-match 'fuzzy' --row-match 'fuzzy'

will provide

S.No id first last age
1 True True True True
2 True True True True
3 True True True True

Scenario 3: Deep row order - Different ordered numerical data

year1 year2 year3 year
751 609 590 930
417 501 441 763
691 621 941 563
179 781 335 225
961 530 433 571

and

year1 year2 year3 year
961 530 433 571
751 609 590 930
691 621 941 563
179 781 335 225
417 501 441 763 
comparesv file1.csv file2.csv --row-match 'deep'
S.No year1 year2 year3 year
1 True True True True
2 True True True True
3 True True True True
4 True True True True
5 True True True True

Scenario n: Unlimited options. Please explore the options below

Description

The first file is considered as the source file. It will be compared against the second file. Refer the below options to finetune the way it works.

Row Match (-rm)

This will define the way how the rows between the files will be identified for comparison

order - This is the default option, This will compare the rows by their position between the files. This can be used if the records in both the files are in same order

fuzzy - This will use fuzzy logic to identify the matching row on second file. This can be used if the records are not in order and most of the data are text.

deep - This will use fuzzy logic to identify the matching row on second file. This can be used if the records are not in order and it has numeric data. This will look for each row in file1 against all the rows in file2 to find a potential match

Column Match (-rm)

This will define the way how the columns between the files will be identified for comparison

exact - This is the default option, This will compare the columns between the files by their headers for an exact match and select it for comparison. eg. 'Age' and 'Age' columns across the files will be selected for comparison.

fuzzy - This will use fuzzy logic to identify the matching column on second file. This can be used if the column headers across the files are not exactly same by somehow closer. eg. 'age' and 'age of student' columns may be selected for comparison.

String Match (-sm)

This will define the way how the textual data is compared.

exact - This is the default option, This will compare the exact text.

fuzzy - This will use fuzzy logic to find if the texts are closer to each other and identifies the match.

Include Additional Rows (-ir)

If the second file contains more rows than the first file, this option will enable the comparison output to include the remaining rows (uncompared ones).

Include Additional Columns (-ic)

If the second file contains more columns than the first file, this option will enable the comparison output to include the remaining columms.

Ignore case (-i)

This option will ignore the case while comparing the strings.

Include Stats (-is)

This option is enabled by default and it outputs the comparison stats (in percentage) on the console

Save Output (-s)

This option will save the result & values comparison in the current directory. This is enabled by default.