-
Notifications
You must be signed in to change notification settings - Fork 0
/
check_duplicates.log
45 lines (38 loc) · 1.8 KB
/
check_duplicates.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
Date: September 2, 2010
59
60 ** Check Duplicates;
61 proc sort data = morn.master_morn_holdings out = tester nodupkey;
62 where ~missing(cusip);
63 by port_date fundid cusip;
64 run;
NOTE: There were 82452724 observations read from the data set MORN.MASTER_MORN_HOLDINGS.
WHERE not MISSING(cusip);
NOTE: 23415 observations with duplicate key values were deleted.
NOTE: The data set WORK.TESTER has 82429309 observations and 49 variables.
NOTE: PROCEDURE SORT used (Total process time):
real time 3:20:18.92
cpu time 3:40.71
65 * maybe description is missing;
66 proc sort data = morn.master_morn_holdings out = tester1 nodupkey;
67 where ~missing(security);
68 by port_date fundid security;
69 run;
NOTE: There were 92556076 observations read from the data set MORN.MASTER_MORN_HOLDINGS.
WHERE not MISSING(security);
NOTE: 12136865 observations with duplicate key values were deleted.
NOTE: The data set WORK.TESTER1 has 80419211 observations and 49 variables.
NOTE: PROCEDURE SORT used (Total process time):
real time 3:51:17.16
cpu time 4:06.80
70 * maybe description is missing;
71 proc sort data = morn.master_morn_holdings out = tester2 nodupkey;
72 where ~missing(marketvalue);
73 by port_date fundid marketvalue;
74 run;
NOTE: There were 92561401 observations read from the data set MORN.MASTER_MORN_HOLDINGS.
WHERE not MISSING(marketvalue);
NOTE: 2270294 observations with duplicate key values were deleted.
NOTE: The data set WORK.TESTER2 has 90291107 observations and 49 variables.
NOTE: PROCEDURE SORT used (Total process time):
real time 2:36:19.49
cpu time 3:57.88