Unemployment Insurance Data Validation

May 2, 2006

REGION 5 ETA WORKFORCE DEVELOPMENT LETTER NO. 005-06

TO: STATE WORKFORCE AGENCY ADMINISTRATORS

FROM:         Byron Zuidema
                  BYRON ZUIDEMA
                  Regional Administrator

SUBJECT: Unemployment Insurance Data Validation

1. Purpose. This Workforce Development Letter provides the status of the Unemployment Insurance (UI) Data Validation (DV) program and informs the States of ETA's decision to suspend submission of validation results during the period May 11, 2006, through March 31, 2007. It explains the reasons for our decision and outlines DV activities during the period between now and March 31, 2007. We ask you to convey this information immediately to your DV coordinators and to the State validation staff in your region

2. References. Unemployment Insurance Program Letter 22-05, April 28, 2005.

3. Links. This Letter is in the Region 5 website archive at: http://doleta.gov/regions/reg05/pages/libraryissuances.cfm

4. Background. In June 2005, ETA released new software for States to use to operate their UI DV programs. This software, which runs on the Sun-UNIX computer supplied by the Department to every State for UI reporting and other related purposes, replaces PC-based software developed by Mathematica Policy Research, Inc. (MPR), the DV technical support contractor. In accordance with policies articulated in Unemployment Insurance Program Letter (UIPL) 22-05, States were to validate reports (1) within the scope of the validation program that either had not been validated for the period ending March 31, 2005, or that did not pass validation, or (2) that must be validated annually because they are used for Government Performance and Results Act (GPRA) measures. The submission mechanism is the same as used for UI required reports. Attachment 1 provides a detailed list of all submissions received using the Sun-based software. As State validators began using the software in regular validation activities and to transmit results to the National Office (NO), various deficiencies became apparent. These involve the software itself, the data transmission, and the structure of the database used to house DV summary results from which the NO determines compliance. We have identified over 40 software bugs (Attachment 2) and another 11 critical enhancements (Attachment 3) that would greatly facilitate validators' use of the software and our ability to administer the DV program. All these changes will be incorporated into Version 1.1, which we expect to release later this calendar year.

The bugs listed in Attachment 2 span nearly the full range of validation activities. The problems found compromise the accuracy of DV results for specific populations and raise questions about the accuracy of a significant portion of results. Hence to ensure States are provided with a reliable tool to conduct data validation we have decided to suspend software-related validation activities for Validation Year (VY) 2007, i.e., validations of UI reports for periods April 1, 2006, through March 31, 2007. This will provide time to develop and thoroughly test the next software release, which will correct all listed bugs and thus produce accurate DV findings. It will also incorporate critical enhancements that will improve software performance and make it more user-friendly.

5. Action. The following summarizes the actions the Department will take and requirements for Validation Years 2006 and 2007.

· For the current Validation Year, VY 2006 (that is, validations of UI reports for periods April 1, 2005, through March 31, 2006):

ü For VY 2006 States are required to:

Ø Submit wage item validations using the DV software.

Ø Submit results from the validation of Tax Performance System (TPS) and Benefits Timeliness and Quality (BTQ) samples in Word documents as e-mail attachments sent to dvrpts@doleta.gov using the attached template (Attachment 4).

Ø Address failures to implement DV by building extract files for all populations or to submit Benefits Timeliness and Quality (BTQ) and Tax Performance System (TPS) validations through the 2007 State Quality Service Plan.

ü States may, at their option:

Ø Complete and submit by May 10, 2006 any validations underway or those they have planned to submit for this Validation Year.

ü DOL Will:

Ø DOL will place on the 3-year validation schedule to be revalidated for VY 2009 all complete population submissions received by May 10, 2006, that pass validation. The next reports for these populations needing to be revalidated will be for the period April 1, 2008, through March 31, 2009. This excludes the populations containing GPRA data elements; these must be validated annually.

Ø Because of the risk that it may have been affected by a software problem, a population that failed validation need not be addressed in the State Quality Service Plan (SQSP).

· For VY 2007:

ü States are required to:

Ø Refrain from submitting any validations of populations using the DV software for this period of refinement.

Ø As in VY 2006, submit results from the validation of TPS and BTQ samples in Word documents as e-mail attachments sent to dvrpts@doleta.gov using the attached template (Attachment 4).

Ø In lieu of submitting validation results during this period, position themselves to resume full-scale validation activities beginning April 1, 2007, by doing the following:

Ø Develop extract files in accordance with the revised record layouts for all DV populations. Populations which have not been validated, or which have failed validation, should receive special attention.

Ø Conduct Independent Validation and Verification (IV&V) activities if FY 2005 SBR funds were received for them.

Ø Update Module 3 data and submit this information to the National Office.

Ø Obtain DV procedural training as needed from MPR and/or software training from the Minnesota Training Center.

Ø When Version 1.1 of the software is released, validators should begin using it immediately to familiarize themselves with it, ready themselves to resume validation of reports for the validation year beginning April 1, 2007, and inform the NO of any functionality problems.

ü DOL will:

Ø Perform extensive testing of the revised DV software. Program and technical staff at the NO will review and alpha test new and corrected functions in the DV software. Alpha testing at the NO will be supplemented by testing at a State site involving MPR, and then by Beta testing in a few States. Identified problems will be corrected and re-tested before the software is released to the States.

Ø Revise both the UI DV Handbook and software users' guides to conform to the revised software.

Ø Update the electronic versions of the State-specific Module 3 for both Benefits and Tax Handbooks and reissue them to each State.

For VY 2008, States will be required to resume validation activity using software version DV WS 1.1. This includes validations that were due for VY 2006 and were not done because of the software problem or other reasons, as well as populations that otherwise would have been due for validation in VY 2007. These are due in 2008 instead of 2007. The suspension of validation submissions for VY 2007 does not affect the timing of validations that are due for validation year 2008, e.g., validation populations due to be submitted in 2008 because they passed in 2005 are still due for 2008.

As soon as possible, ETA will issue a UIPL that formalizes these instructions for the States and also clarifies or amends UIPL 22-05 policy guidance. The policy clarifications and amendments concern procedures for determining pass and fail in populations that contain more than one random sample and instructions for submitting validation results.

6. Contact. Comments or questions concerning the UI Data Validation may be directed to Raymond Koch on 312.596.5440. Comments or questions about the format of this Letter may be directed to Tom Coyne on 312.596.5435

7. Expiration Date. March 31, 2007

8. Attachments.

1. Listing of Sun-based software DV submissions.

2. Listing of Bugs

3. Listing of Enhancements

4. Template for reporting results of TPS and BTQ sample reviews

Attachment 1

Validation Submissions Received as of March 24, 2006

Benefits Report Validation Submissions as of 3-24-2006

State

Pop Type ID Received Reporting Period Groups* Failures* Score*

1 Counts 160 12/08/2005 09/01/2005 09/30/2005 3 0 PASS

2 Counts 150 11/29/2005 07/01/2005 09/30/2005 2 0 PASS

12 Counts 167 12/21/2005 07/01/2005 09/30/2005 3 0 PASS

12 Dollars 168 12/21/2005 07/01/2005 09/30/2005 2 0 PASS

14 Dollars 196 02/09/2006 10/01/2005 12/31/2005 3 0 PASS

Total Submissions by AZ: 5

2 Counts 212 01/26/2006 10/01/2005 12/31/2005 2 0 PASS

6 Counts 214 01/26/2006 12/01/2005 12/31/2005 1 0 PASS

7 Counts 216 01/26/2006 12/01/2005 12/31/2005 1 0 PASS

7 Counts 217 01/26/2006 12/01/2005 12/31/2005 1 0 PASS

Total Submissions by CT: 4

1 Counts 127 08/17/2005 06/01/2005 06/30/2005 3 0 PASS

2 Counts 141 09/20/2005 04/01/2005 06/30/2005 2 0 PASS

3 Counts 133 09/19/2005 04/01/2005 06/30/2005 9 0 PASS

4 Counts 108 08/03/2005 04/01/2005 06/30/2005 9 0 PASS

4 Dollars 109 08/03/2005 04/01/2005 06/30/2005 4 0 PASS

5 Counts 176 12/21/2005 07/01/2005 09/30/2005 12 1 FAIL

Total Submissions by FL: 6

6 Counts 141 03/13/2006 11/01/2005 11/30/2005 1 0 PASS

8 Counts 147 03/17/2006 11/01/2005 11/30/2005 6 2 FAIL

10 Counts 132 02/03/2006 11/01/2005 11/30/2005 1 0 PASS

11 Counts 133 02/03/2006 11/01/2005 11/30/2005 1 0 PASS

Total Submissions by ID: 4

2 Counts 206 02/22/2006 10/01/2005 12/31/2005 2 0 PASS

7 Counts 207 02/23/2006 12/01/2005 12/31/2005 1 0 PASS

12 Counts 204 02/22/2006 10/01/2005 12/31/2005 3 0 PASS

12 Dollars 205 02/22/2006 10/01/2005 12/31/2005 2 0 PASS

Total Submissions by ME: 4

1 Counts 157 09/20/2005 06/01/2005 06/30/2005 3 1 FAIL

2 Counts 160 10/25/2005 04/02/2005 07/01/2005 2 0 PASS

Total Submissions by MN: 2

6 Counts 362 11/09/2005 09/01/2005 09/30/2005 1 0 PASS

7 Counts 364 11/09/2005 09/01/2005 09/30/2005 1 0 PASS

13 Dollars 115 08/29/2005 10/01/2004 12/31/2004 5 0 PASS

Total Submissions by MO: 3

3 Counts 105 08/09/2005 04/01/2005 06/30/2005 9 5 FAIL

3a Counts 112 08/09/2005 04/01/2005 06/30/2005 1 0 PASS

3a Counts 114 09/06/2005 04/01/2005 06/30/2005 1 0 PASS

4 Dollars 152 01/25/2006 10/01/2005 12/31/2005 4 2 FAIL

4 Counts 153 01/25/2006 10/01/2005 12/31/2005 9 7 FAIL

Total Submissions by NH: 5

Page 2

State

Pop Type ID Received Reporting Period Groups* Failures* Score*

1 Counts 242 03/14/2006 09/01/2005 09/30/2005 3 2 FAIL

2 Counts 218 01/20/2006 10/01/2005 12/31/2005 2 2 FAIL

3a Counts 245 03/23/2006 10/01/2005 12/31/2005 1 0 PASS

12 Counts 211 01/06/2006 07/01/2005 09/30/2005 3 0 PASS

12 Dollars 212 01/06/2006 07/01/2005 09/30/2005 2 0 PASS

Total Submissions by NY: 5

6 Counts 133 12/07/2005 09/01/2005 09/30/2005 1 0 PASS

8 Counts 141 01/17/2006 09/01/2005 09/30/2005 6 1 FAIL

Total Submissions by TN: 2

1 Counts 199 07/28/2005 04/01/2005 06/30/2005 3 1 FAIL

1 Counts 371 03/15/2006 10/01/2005 12/31/2005 3 2 FAIL

2 Counts 188 07/22/2005 04/01/2005 06/30/2005 2 0 PASS

3 Counts 189 07/22/2005 04/01/2005 06/30/2005 9 9 FAIL

3 Counts 216 07/29/2005 04/01/2005 06/30/2005 9 6 FAIL

3a Counts 196 07/28/2005 04/01/2005 06/30/2005 1 0 PASS

4 Dollars 190 07/22/2005 04/01/2005 06/30/2005 4 0 PASS

4 Dollars 217 08/01/2005 04/01/2005 06/30/2005 4 0 PASS

4 Dollars 277 08/30/2005 04/01/2005 06/30/2005 4 0 PASS

4 Counts 342 01/25/2006 10/01/2005 12/31/2005 9 2 FAIL

4 Dollars 368 03/08/2006 10/01/2005 12/31/2005 4 0 PASS

4 Dollars 369 03/08/2006 10/01/2005 12/31/2005 4 0 PASS

5 Counts 120 07/15/2005 04/01/2005 06/30/2005 12 2 FAIL

12 Dollars 191 07/26/2005 04/01/2005 06/30/2005 2 0 PASS

12 Counts 346 01/25/2006 10/01/2005 12/31/2005 3 0 PASS

12 Dollars 347 01/25/2006 10/01/2005 12/31/2005 2 0 PASS

13 Dollars 192 07/26/2005 04/01/2005 06/30/2005 5 2 FAIL

13 Dollars 344 01/25/2006 10/01/2005 12/31/2005 5 2 FAIL

14 Dollars 193 07/26/2005 04/01/2005 06/30/2005 3 3 FAIL

14 Dollars 238 08/02/2005 04/01/2005 06/30/2005 3 3 FAIL

14 Dollars 241 08/02/2005 04/01/2005 06/30/2005 3 3 FAIL

14 Dollars 243 08/02/2005 04/01/2005 06/30/2005 3 3 FAIL

14 Dollars 245 08/02/2005 04/01/2005 06/30/2005 3 3 FAIL

14 Dollars 349 01/25/2006 10/01/2005 12/31/2005 3 2 FAIL

Total Submissions by TX: 24

3a Counts 203 02/28/2006 10/01/2005 12/31/2005 1 0 PASS

4 Counts 139 09/30/2005 04/01/2005 06/30/2005 9 6 FAIL

4 Dollars 150 09/30/2005 04/01/2005 06/30/2005 4 3 FAIL

5 Counts 119 08/17/2005 04/01/2005 06/30/2005 12 0 PASS

10 Counts 182 12/21/2005 11/01/2005 11/30/2005 1 0 PASS

13 Dollars 135 09/29/2005 04/01/2005 06/30/2005 5 4 FAIL

14 Dollars 130 09/16/2005 04/01/2005 06/30/2005 3 3 FAIL

Total Submissions by VA: 7

2 Counts 172 01/19/2006 07/01/2005 09/30/2005 2 0 PASS

6 Counts 175 01/19/2006 09/01/2005 09/30/2005 1 0 PASS

7 Counts 174 01/19/2006 09/01/2005 09/30/2005 1 1 FAIL

7 Counts 176 01/19/2006 09/01/2005 09/30/2005 1 1 FAIL

7 Counts 208 02/14/2006 09/01/2005 09/30/2005 1 0 PASS

8 Counts 185 02/01/2006 09/01/2005 09/30/2005 6 1 FAIL

9 Counts 173 01/19/2006 09/01/2005 09/30/2005 5 0 PASS

10 Counts 197 02/10/2006 09/01/2005 09/30/2005 1 0 PASS

11 Counts 201 02/10/2006 09/01/2005 09/30/2005 1 0 PASS

Total Submissions by WY: 9

Total Benefits RV Submissions from All US: 80

*Values calculated at National Office

Page 3

Benefits Data Element Val. Submissions as of 3-24-2006

St Pop Samp Type ID Recvd Reporting Period Univ Size Cases Errs Score

1 100 Random 161 12/08/2005 09/01/2005 09/30/2005 100491 200 60 0 Pass

1 120 Minimum 162 12/08/2005 09/01/2005 09/30/2005 12569 2 6 0

1 110 Random 166 12/20/2005 09/01/2005 09/30/2005 5384 100 30 0 Pass

2 200 Random 151 11/29/2005 07/01/2005 09/30/2005 9715 100 30 0 Pass

12 1200 Random 169 12/21/2005 07/01/2005 09/30/2005 4074 200 60 0 Pass

12 1210 Missing 170 12/21/2005 07/01/2005 09/30/2005 4074 1 2 0

12 1220 Outliers 171 12/21/2005 07/01/2005 09/30/2005 4074 10 10 0

14 1400 Random 197 02/09/2006 10/01/2005 12/31/2005 22802 100 30 0 Pass

14 1410 Outliers 198 02/09/2006 10/01/2005 12/31/2005 22802 10 10 0

14 1420 Missing 199 02/09/2006 10/01/2005 12/31/2005 22802 1 6 0

Total Submissions by AZ: 10

2 200 Random 213 01/26/2006 10/01/2005 12/31/2005 9628 100 30 0 Pass

6 600 Minimum 215 01/26/2006 12/01/2005 12/31/2005 1485 2 2 0

7 700 Minimum 218 01/26/2006 12/01/2005 12/31/2005 149 2 2 0

Total Submissions by CT: 3

1 100 Random 128 08/17/2005 06/01/2005 06/30/2005 351826 200 200 0 Pass

1 110 Random 129 08/17/2005 06/01/2005 06/30/2005 14698 100 30 0 Pass

1 120 Minimum 130 08/17/2005 06/01/2005 06/30/2005 26632 2 24 0

2 200 Random 142 09/20/2005 04/01/2005 06/30/2005 28396 100 30 0 Pass

2 200 Random 143 09/20/2005 04/01/2005 06/30/2005 28396 100 30 0 Pass

3 300 Random 134 09/19/2005 04/01/2005 06/30/2005 103492 200 60 0 Pass

3 301 Missing 135 09/19/2005 04/01/2005 06/30/2005 103492 1 3 0

3 305 Minimum 136 09/19/2005 04/01/2005 06/30/2005 4290 2 6 0

3 310 Minimum 137 09/19/2005 04/01/2005 06/30/2005 3163 2 6 0

3 315 Random 138 09/19/2005 04/01/2005 06/30/2005 3123 100 30 0 Pass

3 320 Random 139 09/19/2005 04/01/2005 06/30/2005 9368 100 30 0 Pass

4 400 Random 110 08/03/2005 04/01/2005 06/30/2005 60286 200 60 0 Pass

4 405 Missing 111 08/03/2005 04/01/2005 06/30/2005 60286 1 8 0

4 410 Outliers 112 08/03/2005 04/01/2005 06/30/2005 58599 10 10 0

4 415 Outliers 113 08/03/2005 04/01/2005 06/30/2005 797109 10 10 0

4 420 Random 114 08/03/2005 04/01/2005 06/30/2005 27998 100 30 0 Pass

4 425 Outliers 115 08/03/2005 04/01/2005 06/30/2005 956 10 10 0

4 435 Random 116 08/03/2005 04/01/2005 06/30/2005 2147 100 30 0 Pass

4 440 Minimum 117 08/03/2005 04/01/2005 06/30/2005 21102 2 4 0

4 445 Minimum 118 08/03/2005 04/01/2005 06/30/2005 191 2 4 0

5 500 Random 177 12/21/2005 07/01/2005 09/30/2005 77928 100 100 2 Pass

5 501 Missing 178 12/21/2005 07/01/2005 09/30/2005 77928 1 36 0

5 505 Outliers 179 12/21/2005 07/01/2005 09/30/2005 77928 10 10 0

5 510 Minimum 180 12/21/2005 07/01/2005 09/30/2005 7 2 3 0

5 520 Random 181 12/21/2005 07/01/2005 09/30/2005 451 100 30 0 Pass

5 500 Random 182 12/21/2005 07/01/2005 09/30/2005 77928 100 100 2 Pass

5 501 Missing 183 12/21/2005 07/01/2005 09/30/2005 77928 1 36 0

5 505 Outliers 184 12/21/2005 07/01/2005 09/30/2005 77928 10 10 0

5 510 Minimum 185 12/21/2005 07/01/2005 09/30/2005 7 2 3 0

5 520 Random 186 12/21/2005 07/01/2005 09/30/2005 451 100 30 0 Pass

Total Submissions by FL: 30

5 520 Random 107 02/08/2006 10/01/2005 12/31/2005 55 100 30 5 Fail

Total Submissions by MD: 1

Page 4

St Pop Samp Type ID Recvd Reporting Period Univ Size Cases Errs Score

1 100 Random 158 09/22/2005 06/01/2005 06/30/2005 151911 200 200 6 Pass

2 200 Random 159 10/25/2005 04/02/2005 07/01/2005 12460 100 30 0 Pass

Total Submissions by MN: 2

6 600 Minimum 361 11/09/2005 09/01/2005 09/30/2005 2547 2 2 0

7 700 Minimum 363 11/09/2005 09/01/2005 09/30/2005 361 2 2 0

13 1300 Random 112 08/29/2005 10/01/2004 12/31/2004 18042 100 30 0 Pass

13 1310 Missing 113 08/29/2005 10/01/2004 12/31/2004 18042 1 16 0

13 1320 Outliers 114 08/29/2005 10/01/2004 12/31/2004 18042 10 10 5

Total Submissions by MO: 5

3 300 Random 106 08/09/2005 04/01/2005 06/30/2005 8202 200 60 0 Pass

3 301 Missing 107 08/09/2005 04/01/2005 06/30/2005 8202 1 2 0

3 305 Minimum 108 08/09/2005 04/01/2005 06/30/2005 908 2 6 0

3 310 Minimum 109 08/09/2005 04/01/2005 06/30/2005 209 2 5 0

3 315 Random 110 08/09/2005 04/01/2005 06/30/2005 194 100 30 0 Pass

3 320 Random 111 08/09/2005 04/01/2005 06/30/2005 516 100 30 30 Fail

3a 360 Minimum 113 08/09/2005 04/01/2005 06/30/2005 119 2 4 0

3a 350 Random 115 09/06/2005 04/01/2005 06/30/2005 2863 200 200 1 Pass

4 400 Random 143 01/18/2006 10/01/2005 12/31/2005 4173 200 200 1 Pass

4 435 Random 144 01/18/2006 10/01/2005 12/31/2005 75 100 30 5 Fail

4 410 Outliers 145 01/25/2006 10/01/2005 12/31/2005 3968 10 10 0

4 425 Outliers 146 01/25/2006 10/01/2005 12/31/2005 153 10 10 0

4 415 Outliers 147 01/25/2006 10/01/2005 12/31/2005 45453 10 10 0

4 440 Minimum 148 01/25/2006 10/01/2005 12/31/2005 831 2 4 0

4 450 Minimum 149 01/25/2006 10/01/2005 12/31/2005 18 2 2 0

4 450 Minimum 150 01/25/2006 10/01/2005 12/31/2005 18 2 2 0

4 445 Minimum 151 01/25/2006 10/01/2005 12/31/2005 8 2 2 0

4 405 Missing 157 02/01/2006 10/01/2005 12/31/2005 4173 1 6 1

4 420 Random 164 02/07/2006 10/01/2005 12/31/2005 4936 100 100 6 Pass

4 435 Random 165 02/21/2006 10/01/2005 12/31/2005 75 100 30 0 Pass

8 800 Random 166 03/16/2006 10/01/2005 12/31/2005 779 200 60 8 Fail

8 810 Missing 167 03/16/2006 10/01/2005 12/31/2005 187 1 3 0

8 820 Outliers 168 03/16/2006 10/01/2005 12/31/2005 779 10 10 1

12 1200 Random 127 11/08/2005 07/01/2005 09/30/2005 750 200 60 8 Fail

12 1200 Random 128 11/08/2005 07/01/2005 09/30/2005 750 200 60 8 Fail

12 1210 Missing 129 11/08/2005 07/01/2005 09/30/2005 750 1 2 0

12 1220 Outliers 130 11/08/2005 07/01/2005 09/30/2005 750 10 10 1

12 1210 Missing 131 11/08/2005 07/01/2005 09/30/2005 750 1 2 0

13 1300 Random 132 11/16/2005 07/01/2005 09/30/2005 446 100 30 5 Fail

13 1310 Missing 133 11/16/2005 07/01/2005 09/30/2005 446 1 3 1

13 1320 Outliers 134 11/16/2005 07/01/2005 09/30/2005 446 10 10 6

14 1400 Random 137 11/30/2005 07/01/2005 09/30/2005 2942 100 100 23 Fail

14 1410 Outliers 138 11/30/2005 07/01/2005 09/30/2005 2942 10 10 2

14 1420 Missing 139 11/30/2005 07/01/2005 09/30/2005 2942 1 2 0

14 1400 Random 154 01/25/2006 07/01/2005 09/30/2005 2942 100 100 23 Fail

14 1410 Outliers 155 01/25/2006 07/01/2005 09/30/2005 2942 10 10 2

14 1420 Missing 156 01/25/2006 07/01/2005 09/30/2005 2942 1 2 0

Total Submissions by NH: 37

3a 350 Random 246 03/23/2006 10/01/2005 12/31/2005 119464 200 60 8 Fail

3a 360 Minimum 247 03/23/2006 10/01/2005 12/31/2005 2515 2 6 3

12 1200 Random 213 01/06/2006 07/01/2005 09/30/2005 15624 200 60 0 Pass

12 1210 Missing 214 01/06/2006 07/01/2005 09/30/2005 15624 1 4 0

12 1220 Outliers 215 01/06/2006 07/01/2005 09/30/2005 15624 10 10 3

Total Submissions by NY: 5

Page 5

St Pop Samp Type ID Recvd Reporting Period Univ Size Cases Errs Score

12 1210 Missing 175 02/10/2006 10/01/2005 12/31/2005 2024 1 2 0

14 1400 Random 176 02/10/2006 10/01/2005 12/31/2005 10416 100 30 0 Pass

14 1410 Outliers 177 02/10/2006 10/01/2005 12/31/2005 10416 10 10 0

14 1420 Missing 178 02/10/2006 10/01/2005 12/31/2005 10416 1 5 0

Total Submissions by PR: 4

6 600 Minimum 134 12/07/2005 09/01/2005 09/30/2005 1586 2 2 0

8 810 Missing 138 01/12/2006 09/01/2005 09/30/2005 129 1 3 0

8 820 Outliers 139 01/12/2006 09/01/2005 09/30/2005 1734 10 10 1

8 820 Outliers 140 01/12/2006 09/01/2005 09/30/2005 1734 10 10 1

8 810 Missing 142 01/17/2006 09/01/2005 09/30/2005 129 1 3 0

8 810 Missing 143 01/17/2006 09/01/2005 09/30/2005 129 1 3 0

8 820 Outliers 144 01/17/2006 09/01/2005 09/30/2005 1734 10 10 1

Total Submissions by TN: 7

1 120 Minimum 372 03/22/2006 10/01/2005 12/31/2005 516440 2 6 6

1 110 Random 373 03/23/2006 10/01/2005 12/31/2005 31913 100 30 0 Pass

2 200 Random 278 09/01/2005 04/01/2005 06/30/2005 36789 100 30 0 Pass

3 321 Missing 330 12/08/2005 04/01/2005 06/30/2005 7190 1 1 0

3 300 Random 331 12/09/2005 04/01/2005 06/30/2005 159813 200 60 16 Fail

3 301 Missing 332 12/09/2005 04/01/2005 06/30/2005 159813 1 4 3

3 305 Minimum 333 12/09/2005 04/01/2005 06/30/2005 3219 2 6 1

3 310 Minimum 334 12/09/2005 04/01/2005 06/30/2005 3686 2 6 4

3 321 Missing 335 12/09/2005 04/01/2005 06/30/2005 7190 1 1 0

3a 350 Random 280 09/02/2005 04/01/2005 06/30/2005 40347 200 23 0

3a 360 Minimum 281 09/02/2005 04/01/2005 06/30/2005 1306 2 6 1

3a 350 Random 339 01/23/2006 04/01/2005 06/30/2005 40347 200 23 0

3a 360 Minimum 340 01/23/2006 04/01/2005 06/30/2005 1306 2 6 1

4 400 Random 374 03/23/2006 10/01/2005 12/31/2005 94923 200 60 0 Pass

4 405 Missing 375 03/23/2006 10/01/2005 12/31/2005 94923 1 9 0

4 410 Outliers 376 03/23/2006 10/01/2005 12/31/2005 91977 10 10 2

4 425 Outliers 377 03/24/2006 10/01/2005 12/31/2005 334 10 10 0

4 425 Outliers 379 03/24/2006 10/01/2005 12/31/2005 334 10 10 0

5 500 Random 308 10/27/2005 04/01/2005 06/30/2005 126567 100 30 5 Fail

5 501 Missing 311 11/01/2005 04/01/2005 06/30/2005 126567 1 39 14

5 505 Outliers 312 11/01/2005 04/01/2005 06/30/2005 126567 10 10 5

5 520 Random 336 01/12/2006 04/01/2005 06/30/2005 10 100 10 0

6 600 Minimum 321 11/18/2005 06/01/2005 06/30/2005 9604 2 2 0

7 700 Minimum 329 11/23/2005 06/01/2005 06/30/2005 1550 2 2 2

8 800 Random 322 11/23/2005 06/01/2005 06/30/2005 10628 200 60 0 Pass

8 810 Missing 323 11/23/2005 06/01/2005 06/30/2005 2334 1 7 0

8 820 Outliers 324 11/23/2005 06/01/2005 06/30/2005 10628 10 10 0

9 900 Random 325 11/23/2005 04/01/2005 06/30/2005 4568 100 30 0 Pass

9 920 Outliers 326 11/23/2005 04/01/2005 06/30/2005 4568 10 10 0

10 1000 Outliers 328 11/23/2005 06/01/2005 06/30/2005 7859 10 10 3

11 1100 Outliers 327 11/23/2005 06/01/2005 06/30/2005 1576 10 10 1

12 1200 Random 246 08/08/2005 04/01/2005 06/30/2005 27105 200 60 10 Fail

12 1210 Missing 247 08/08/2005 04/01/2005 06/30/2005 27105 1 3 0

12 1200 Random 361 03/01/2006 10/01/2005 12/31/2005 20845 200 60 0 Pass

12 1210 Missing 362 03/01/2006 10/01/2005 12/31/2005 20845 1 3 0

12 1220 Outliers 363 03/01/2006 10/01/2005 12/31/2005 20845 10 10 1

13 1320 Outliers 309 10/28/2005 04/01/2005 06/30/2005 67680 10 10 5

13 1310 Missing 310 10/28/2005 04/01/2005 06/30/2005 67680 1 4 2

Total Submissions by TX: 38

Page 6

St Pop Samp Type ID Recvd Reporting Period Univ Size Cases Errs Score

3a 350 Random 204 02/28/2006 10/01/2005 12/31/2005 22585 200 60 13 Fail

3a 360 Minimum 205 02/28/2006 10/01/2005 12/31/2005 864 2 6 0

4 400 Random 140 09/30/2005 04/01/2005 06/30/2005 24821 200 60 11 Fail

4 405 Missing 141 09/30/2005 04/01/2005 06/30/2005 24821 1 6 3

4 410 Outliers 142 09/30/2005 04/01/2005 06/30/2005 22956 10 10 3

4 415 Outliers 143 09/30/2005 04/01/2005 06/30/2005 273047 10 10 0

4 420 Random 144 09/30/2005 04/01/2005 06/30/2005 56973 100 30 26 Fail

4 425 Outliers 145 09/30/2005 04/01/2005 06/30/2005 1110 10 10 10

4 435 Random 146 09/30/2005 04/01/2005 06/30/2005 1380 100 30 12 Fail

4 440 Minimum 147 09/30/2005 04/01/2005 06/30/2005 12593 2 4 4

4 445 Minimum 148 09/30/2005 04/01/2005 06/30/2005 66 2 4 4

4 450 Minimum 149 09/30/2005 04/01/2005 06/30/2005 260 2 4 4

5 500 Random 120 08/17/2005 04/01/2005 06/30/2005 24587 100 30 0 Pass

5 501 Missing 121 08/17/2005 04/01/2005 06/30/2005 24587 1 35 3

5 505 Outliers 122 08/17/2005 04/01/2005 06/30/2005 24587 10 10 0

10 1000 Outliers 183 12/21/2005 11/01/2005 11/30/2005 1880 10 10 0

13 1300 Random 136 09/29/2005 04/01/2005 06/30/2005 17667 100 30 7 Fail

13 1310 Missing 137 09/29/2005 04/01/2005 06/30/2005 17667 1 14 5

13 1320 Outliers 138 09/29/2005 04/01/2005 06/30/2005 17667 10 10 7

14 1400 Random 131 09/16/2005 04/01/2005 06/30/2005 22168 100 30 30 Fail

14 1410 Outliers 132 09/16/2005 04/01/2005 06/30/2005 22168 10 10 10

14 1420 Missing 133 09/16/2005 04/01/2005 06/30/2005 22168 1 6 6

14 1410 Outliers 134 09/16/2005 04/01/2005 06/30/2005 22168 10 10 10

Total Submissions by VA: 23

1 100 Random 186 02/03/2006 09/01/2005 09/30/2005 5898 200 60 0 Pass

1 110 Random 187 02/03/2006 09/01/2005 09/30/2005 677 100 30 0 Pass

2 200 Random 167 01/19/2006 07/01/2005 09/30/2005 633 100 30 0 Pass

6 600 Minimum 168 01/19/2006 09/01/2005 09/30/2005 96 2 2 0

7 700 Minimum 171 01/19/2006 09/01/2005 09/30/2005 8 2 2 0

7 700 Minimum 207 02/14/2006 09/01/2005 09/30/2005 7 2 2 0

8 800 Random 183 02/01/2006 09/01/2005 09/30/2005 92 200 60 0 Pass

8 820 Outliers 184 02/01/2006 09/01/2005 09/30/2005 92 10 10 0

9 900 Random 169 01/19/2006 09/01/2005 09/30/2005 10 100 10 0

9 920 Outliers 170 01/19/2006 09/01/2005 09/30/2005 10 10 10 1

10 1000 Outliers 198 02/10/2006 09/01/2005 09/30/2005 87 10 10 0

11 1100 Outliers 200 02/10/2006 09/01/2005 09/30/2005 6 10 6 0

Total Submissions by WY: 12

Total Benefits DEV Submissions from All US: 177

Page 7

Tax Report Validation Submissions as of 3-24-2006

State

Pop Type ID Received Items Failures Score

5 Counts 114 08/22/2005 01/01/2005 03/31/2005 3 0 PASS

5 Dollars 115 08/22/2005 01/01/2005 03/31/2005 8 0 PASS

Total Submissions by CT: 2

5 Counts 193 02/01/2006 07/01/2005 09/30/2005 3 0 PASS

5 Dollars 194 02/01/2006 07/01/2005 09/30/2005 8 0 PASS

Total Submissions by FL: 2

5 Counts 233 02/17/2006 04/01/2005 06/30/2005 3 1 FAIL

Total Submissions by NY: 1

1 Counts 272 08/23/2005 04/01/2005 06/30/2005 3 0 PASS

2 Counts 237 08/02/2005 04/01/2005 06/30/2005 7 0 PASS

2 Counts 256 08/10/2005 01/01/2005 03/31/2005 7 3 FAIL

2 Counts 259 08/22/2005 01/01/2005 03/31/2005 7 0 PASS

3 Counts 273 08/23/2005 04/01/2005 06/30/2005 8 5 FAIL

3 Counts 274 08/23/2005 04/01/2005 06/30/2005 8 5 FAIL

4 Dollars 266 08/22/2005 04/01/2005 06/30/2005 10 9 FAIL

4 Dollars 267 08/22/2005 04/01/2005 06/30/2005 10 9 FAIL

4 Dollars 271 08/23/2005 04/01/2005 06/30/2005 10 2 FAIL

4 Dollars 366 03/01/2006 10/01/2005 12/31/2005 10 1 FAIL

5 Counts 305 09/29/2005 04/01/2005 06/30/2005 3 1 FAIL

5 Dollars 306 09/29/2005 04/01/2005 06/30/2005 8 0 PASS

5 Counts 319 11/18/2005 04/01/2005 06/30/2005 3 0 PASS

5 Dollars 320 11/18/2005 04/01/2005 06/30/2005 8 0 PASS

Total Submissions by TX: 14

Total Tax RV Submissions from All US: 19

Tax Data Element Validation Submissions as of 3-24-2006

State

Pop ID Sample Received Reporting Period Cases Errors % Error* Score*

5 175 10400 12/19/2005 07/01/2005 09/30/2005 8 0 0.0% PASS

5 195 10400 02/01/2006 07/01/2005 09/30/2005 8 0 0.0% PASS

Total Submissions by FL: 2

5 234 10400 02/17/2006 04/01/2005 06/30/2005 8 0 0.0% PASS

Total Submissions by NY: 1

1 142 10000 12/15/2005 04/01/2005 06/30/2005 4 0 0.0% PASS

Total Submissions by PR: 1

1 355 10000 02/17/2006 04/01/2005 06/30/2005 4 1 25.0% FAIL

2 356 10100 02/17/2006 01/01/2005 03/31/2005 20 0 0.0% PASS

3 337 10200 01/12/2006 04/01/2005 06/30/2005 8 0 0.0% PASS

3 357 10200 02/17/2006 04/01/2005 06/30/2005 8 0 0.0% PASS

4 359 10300 02/17/2006 04/01/2005 06/30/2005 24 6 25.0% FAIL

5 358 10400 02/17/2006 04/01/2005 06/30/2005 8 0 0.0% PASS

Total Submissions by TX: 6

Total Tax DEV Submissions from All US: 10

* Calculated by National Office.

Attachment 2

Benefits Software Bugs and Fixes
	*Population*	*Description of Problem*	*Effect*	*Comment/Fix*
1	*All*	Comments received from States arrive garbled at DOL		Increase size of the field and ensure that comments display properly.
2	1, 3, 9, 14	Errors/typos in Extract File layouts available off the Population link		Correct errors & typos.
3	1, 2, 3, 3a, 4, 5, 8, 9, 12, 13, 14	In random samples, when user saves the second tier if he/she has not completed entering results for at least one whole row, two errors occur: 1. all entered "fails" in tier 2 get added to tier 1 summary and results are calculated for tier 1 2. the software displays the screen for tier 1 instead of tier 2 as it should	Could cause inadvertent failure, or lock user out of Stage 2 sample, forcing him to start over with sample entry/review.	Validators will be able to view status of sample (including Pass/Fail) by using SAVE; when Tier 1 is complete, validator will have choice of submitting or going to Tier 2 without hitting SUBMIT. Once in Tier 2, Tier 1 records are frozen until validator selects to return to Tier 1 of sample. No results from incomplete rows will be added to totals (all samples will be made to work this way.)
4	1, 2, 3, 3a, 4, 5, 8, 9, 12, 13, 14	At present the software changes all fields to "pass" when the "Check Row" box is checked, and double-clicking "Check All" box clears all data in sample.	Could cause inadvertent sample pass if rows with errors are cleared and replaced with passes before sample is submitted.	"Check Row" and "Check All" buttons replaced by "Pass" and "Pass All" buttons. "Pass" button changes only blank fields to pass. Separate "Clear" and "Clear All" buttons with warning to user. When user is in Tier 2, "Clear All" button only clears data in that Tier.
5	3	Software rejecting Prior Qtr transactions (3.38-3.45--CWC Prior Quarter claims and UI New Claims filed in Prior Quarter)	Will cause RV failure if these transactions are present.	Change edit to allow Prior Quarter dates for CWC Prior Quarter and UI New Claims filed in Prior Quarter.
6	3	Software is placing sub population 3.31 in validation count for line 102-6 (UCFE Transitional), instead of Line 101-6 (UI transitional). This error causes an undercount for line 101-6 and an over count for line 102-6.	Will cause RV failure if these transactions are present.	Count for 5159 101-6 will be changed to be sum of subpops 3.25-3.31; count for 102-6 will be changed to be 3.32 only.
7	3	Software is rejecting CWC Insufficient Monetary transactions if date of original monetary is missing	Will cause RV failure if these transactions are present.	Software will allow Null or None for this date field; by definition there is no CWC monetary. Handbook changed to make this clear.
8	3a	Import Messages for this pop do not appear automatically; user must select "Get More Messages."		Functionality will be made comparable to other Populations
9	4	Attempting to select Sample 425 (Outlier sample: adjusted pmts.) results in error message, "An unexpected error occurred."	Sample cannot be completed (compliance not affected: not a random sample)	Function will be fixed.
10	4	Extract File for population 4 will be reconfigured to include three months of quarterly data, but only the most recent month of monthly data. This should result in extract file size for pop 4 which the software can handle.	This would affect ability of New York and California to validate Population 4. Smaller States not affected; Texas successfully validated Population 4 with file of 1.3 million records.	We will follow the MPR handbook and software procedure. Software will pick up the last month's counts in the quarter for the 5159, 9050, and 9051 reports for the quarter used for the 586. This way the latest data will be validated. Three months worth of data are only needed to validate the 586 report. This should reduce file sizes in California to a manageable 2 million records or fewer.
11	4	Software is rejecting as duplicates some adjustments and transactions that have same SSN, WE Date, total $ amount for columns 10-14, and mail date.		For all Benefits and Tax Populations, implement duplicate detection criteria worked out with MPR and forwarded on 1/27/2006. See #3 in General Operational Issues, below.
12	5	Duplicate detection criteria refers to SSN and Issue ID. Because Issue ID is not mandatory, software cannot sometimes identify duplicate nonmon records.		Software will implement alternative duplicate detection criteria. The new process will allow validator to assess potential duplicates and reenter legitimate nonmons.
13	5	Some States have nonmon unique ID of > 20 characters (FL's is 35-characters). Software allows only 20. Results in truncation/ duplicates.		Implement MPR recommended field characteristics, which include 50 characters for this unique ID field.
14	5	Software counts number of multi-claimant determinations instead of # claimants involved in Multiclaimant determinations. MPR software had same problem.	Cannot validate 207 lines 101 and 102, col. 4, 5, 6.	Because resolving this would involve changing State extract file, and because MPR software also had this deficiency, it is considered a long-term enhancement. Steps will be taken to ensure that no State fails Population 5 because of this problem.
15	5	View Report Counts for 9052 and 9053 show c193 for Multi-clmt Labor Detection and c194 for MC Other Detection. Cells should be 194 and 195, respectively. Labels should read "Multi-Claimant Labor Dispute" and Multi-Claimant Other." Software retrieving cells 193 and 194 for 9053 report instead of 194 and 195.	Could cause population to fail if State has this type of determinations.	Change labels to c194 "Multi-Claimant Labor Dispute" and "c195 Multi-Claimant Other." Ensure right cells being retrieved in accord with corrected labels.
16	5	Get error message when attempting to select sample 520 (random, single claimant redets) if file contains redets.	Cannot complete compliance requirements for Population 5.	Correct software to select this sample.
17	6	Software rejects multi-claimant appeals for which # claimants = 1.	Could cause population to fail if State creates one record for each claimant in a multi-claimant appeal.	Follow coding of Pop 7, which does accept multi-claimant appeals for which # claimants =1.
18	6	Software counts # multiclaimant appeals not # claimants, when producing validation count for 5130 line 200-10.	Could cause population to fail if State has many multi-claimant appeals.	Software will sum counts in field 6 for multi-claimant appeals, not count # entries.
19	7	Software counts # multiclaimant appeals not # claimants, when producing validation count for 5130 line 200-11.	Could cause population to fail if State has many multi-claimant appeals.	Software will sum counts in field 6 for multi-claimant appeals, not count # entries.
20	8	Software counts # multiclaimant appeals not # claimants, when producing validation count for 5130 line 210-10.	Could cause population to fail if State has many multi-claimant appeals.	Software will sum counts in field 9 for multi-claimant appeals, not count # entries.
21	8	Software retrieves 9055L cell 2 and 3 (Lower Authority Appeals Pending <= days and pending 26-40 days instead of 9054L cells 2 and 3 (UI Intra Total and UI Inter total) when giving Reported Count.	Could cause population to fail.	Correct code to retrieve correct report count.
22	9	Software counts # multiclaimant appeals not # claimants, when producing validation count for 5130 line 210-11.	Could cause population to fail if State has many multi-claimant appeals.	Software will sum counts in field 9 for multi-claimant appeals, not count # entries.
23	12	Sample 1200 is drawn randomly but the cases presented to the validator for review are arrayed by size of overpayment. Thus a 60-case Stage 1 sample contains only the 60 lowest overpayment values and does not represent the universe.	Stage 1 sample will not be representative; full sample will be representative.	We will check to ensure that sample is randomly drawn and make sure that cases presented for validator are not arrayed.
24	13	In View Validation Counts, references in ETA Report/Line/Column for all subpops should be to ETA 227C, not ETA 227B; and for subpopulations 13.26-13.34, references should be to column (14) instead of (13) When you click on each subpopulation for detailed records, heading of table refers to "Overpayments Established" instead of "Overpayment Reconciliation Activities." Header that is displayed when the View Reported Dollars Screen prints says "View Reported *Counts* for Population 13"		References will be corrected.
25	14	Software is giving all cases an age of 0 when it creates sample worksheet.	Will cause every sample to fail.	This field will be changed to contain age of the receivable, computed as difference in days between end of report quarter and date overpayment was established (field 4).
26	1, 2, 3, 3a, 4, 5, 8, 9, 12, 13, 14	Revise sampling score algorithm when universe is no more than first- or second-stage random sample.	Samples from universes smaller than stated sample size do not get a score.	Software will implement a small-sample algorithm provided by MPR with appropriate pass/fail cut-off values.
27	1, 2, 3, 3a, 4, 5, 8, 9, 12, 13, 14	Software allows incomplete results to be submitted for random samples. Errors in uncompleted rows get added to total errors and imputed to completed cases. When errors from uncompleted rows in the second tier get added to completed cases in the first tier, it may result in sample failing at first tier when it shouldn't.	Could cause unwarranted failure of sample.	The software will contain edits that do not allow users to submit incomplete results for any random or nonrandom samples: i.e., all sampled cases must be completed; all elements in a sample must be checked; and inconclusive results at first stage of random samples must not submit. Software will save errors from incomplete cases but not add them to totals. See #3 above.

Tax Software Bugs and Fixes
	*Population*	*Description of Problem*	*Effect*	*Comment/Fix*
1	Help link	File layout for Pop 3 user gets after clicking on the Population link is wrong. About half of the columns are taken from Population 1.	Forces validator to go to alternative source (e.g., MPR software guide) for correct layout.	Will be corrected.
2	1	Last 2 columns of records displayed in View Errors listing are blank. Verified that records in extract file have data.	Forces validator to go to extract file to see correct display of error cases.	Will be corrected.
3	1	FIV samples are 1^st two cases appearing in subpops 1.1 and 1.2, not randomly selected. Depending on how extract file is built, software may display multiple copies of selected case as FIV sample.	May not have adverse effect but should be randomly selected.	FIV cases will be randomly selected and be only two per subpop.
4	1	Help screen says that reactivation date (field 6) is mandatory, Activation Processing date (field 8) is not. However, software rejects transactions for which field 8 is blank with an error message.	Could cause cases to be rejected as errors and affect validation results; State must correct the file.	Software will conform to the new layout/Handbook specification that Field 8 is mandatory. For all employers with liability met date after 12/31/2002, the filter for pending employers requires that date in either field 6 or 8 be on or after liability date (applied to all employers with liability met date after 12/31/2002) so one or the other must be present. Help screen will be changed.
5	2	DEV sorts 2.1 and 2.2 not working properly. The sorts bring up Population 1 data instead of Population 2.	Invalid results for States where these sorts are applicable.	Will be corrected.
6	3	Columns of records displayed in View Errors file are mixed up: Predecessor Acct # appears in Reactivation Date field	Forces validator to go to extract file to see correct display of error cases.	Will be corrected.
7	3	Software is allocating no transactions to subpops 3.2, 3.3, 3.5, and 3.6 because it is not calculating time lapse. Hence, all New and all Successor Determinations get time lapse = 0 and appear in Timely, <181 days, and total categories.	Renders Population RV invalid.	Will be corrected.
8	3	DV WS 1.0 computes the Status Determinations line by summing all seven lines above it, which involves double-counting the time-lapsed counts for New and Successor Determinations. The MPR software sums only the total counts (lines 586/301-14, 17, 20).	Total line involves double counting. Probably of minor significance, if any, as validation counts and reported counts are summed the same way.	Will be corrected to conform to the MPR model.
9	4	When errors occur, View Errors displays amounts in the wrong columns. It is two columns off. 'Age of Receivable' now shows in the 'Removed' column'; 'Balance at End of RQ' appears in the 'Uncollectible' column.	Forces validator to go to extract file to see correct display of error cases.	Will be corrected.
10	4	Load aborted when it encountered a row with a blank field that should not have been null, instead of removing transaction to error file and continuing load.	Has required at least one validator to seek assistance from National Office to get an extract file to load.	We will ensure that unusable transactions are removed to the errors instead of causing the load to abort.
11	5	Software rejects transactions in which the Employer Size (field 4) and Change Audit (field 5) are more than single character. Some States have secondary codes for these fields	Has required at least one validator to seek assistance from National Office to get an extract file to load.	Software specifications amended to allow larger fields, following model of MPR software. These are included in revised field specifications for extract files.
12	5	for FIV sample: § Software picks first two cases in each subpop not at random § Employer size field says "null" but field must have value of S or L § The pass-fail box next to employer size has "> in it.	Validators cannot validate the S/L field in the FIV samples.	All FIV cases will be selected randomly. Errors will be corrected.
13	5	Software rejects certain transactions with very large pre- or post-audit total wages with error message "Reconciliation Amount Not Zero."	Can cause RV failure by rejecting large employer (Total Wages >$20 million) transactions.	The wage value fields will be defined as double precision instead of float.
14	1 - 5	An FIV sampled case with even one cell completed is treated as a complete case. Software will submit a set of cases as long as each case has at least one entry.	May permit submission of incomplete samples and allow some FIV samples to pass inadvertently.	Functionality for FIV samples will be made the same as for Benefits minimum samples: The software will not allow users to submit incomplete results for any samples, i.e., all sampled cases must be completed; all elements in a sample must be checked. Software will save errors from incomplete cases but not add them to totals.

General Operational Issues
1	Users can be timed out after 30 minutes even if they continue to enter data unless they (a) save results or (b) go to another screen. Most likely to be a problem when entering data for a Benefits Random sample.	The next version of the software will keep a session alive as long as the user is actively using a function on the screen, e.g. entering data, and not necessarily saving or submitting or moving to another screen
2	Population loads or sample draws may abort when a row containing a field that does not contain a value required, or encounters a value in a field that must be blank.	The entire application will be reviewed to ensure that encountering an inappropriate transaction does not abort load or sample draw; but simply puts the row into error screen with appropriate error message.
3	MPR has issued revised duplicate detection criteria for most Benefits and Tax Populations worked out in conjunction with UI staff.	For all Benefits and Tax Populations, implement duplicate detection criteria worked out with MPR and forwarded on 1/27/2006, revised on 3/23/2006 (Benefits Population 4).

Attachment 3
Enhancements That Will Be Included In DV WS 1.1
Item #	Type	Population	Validation Type	Enhancement	Comments
1	Benefits and Tax	All	All	Add a load ID to all submissions. The load ID will be used to identify submissions that are coming from the same extract file.	Will enable NO to link RV and DEV results to same extract file (essential for determining whether Tax populations pass)
2	Benefits and Tax	All and Wage Items	All	Add submission date stamp to all submissions. The submission date stamp will be used to distinguish among submissions made or picked up on the same day.	Will enable NO to differentiate multiple submissions on same day and thus to use the last one in preparing results reports.
3	Tax	All	RV, Sorts and FIVs	Tax RV, Sorts and FIVs should be submitted to the National Office as a set. First check that all tests have been completed and then give the user the option to submit the population as a set.	This change enforces the Tax validation logic that an extract file must first pass all DEV tests before being considered reliable for RV.
4	General	All and Wage Items	All	Increase comments size. Provide separate comments for DEV and RV, and Wage Items accessible through the respective screens. Each sample in benefits should have its own comments. All comments should be linked to the set of records to which they pertain.	This change will correct the Benefits Bug #1, garbled comments.
5	Benefits	All	RV	Add numbers to groups in benefits. Each group line should have a group number that should be displayed on the RV screen and stored as a field to be brought to the NO database.	This will simplify preparation of DV results reports for the National Office and clarify RV results for the validators.
6	Tax	1-4	Sorts	Add a N/A check box at the beginning of the row for each sort that the user can check when the sort cannot be performed in the State. When this box is checked the fields "# of Cases" and "# of Errors" for that sort will be disabled and set to null, and the status for that sort will display N/A (see enhancement # 7).	At present there is no way for NO to know whether sorts are required and thus whether passing DEV requires passing sorts or not. This will convey that information via the DV software.
7	Tax	1-4	Sorts	Display status, i.e. Pass, Fail or N/A, for each sort at the end of the line of each row. The status is "N/A" when the user checks the N/A box (see enhancement # 5), "Pass" when the "Pct of Errors" column is less than or equal to 2, and "Fail" when the "Pct of Errors" column is greater than 2.	This will present all sort information on the screen for the validator.
8	Tax	1-4	Sorts	Software will retrieve values in "# of Cases" column. The value will equal the number of records sorted; it will be set to null when the N/A box is checked (see enhancement # 6).	The validator will no longer have to retrieve and enter the number of transactions sorted; risk of errors is minimized.
9	Tax	Wage Items		Display Pass/Fail score for each line	The validator can tell which wage items pass as soon as data are entered.
10	General			When a population extract file is loading, the software will (a) lock import for other populations and (b) lock other functions for the population being loaded, but not (c) functions other than load for other populations.	This will reduce the chance of errors, e.g., seeing incomplete validation counts compiled from an incomplete load or the chance of mixed load messages from simultaneous loads.
11	Benefits and Tax	Benefits 4 and 12; Tax 5	RV	Combine Dollars and Counts into one "Report Validation" screen and submit them together when results are submitted.	Because determining compliance requires that all aspects of a report pass RV, this ensures that validator sees entire picture at time of validation and makes sure that NO receives RV results as a set.

Attachment 4

Templates for Reporting Results of Validations of Quality Sampling Reviews

State:

Nonmonetary Quality Sample (Benefits Pop 5)

Year/Quarter: YYYY:Q

Universe: XXX,XXX

9052 Count: XXX,XXX

Difference as % of 9052 Count:

Sampling Method: (Interval or Randomized File)

Problems/Comments:

State:

Lower Authority Appeals Quality Sample (Benefits Pop 8)

Year/Quarter: YYYY:Q

Universe: XXX,XXX

9054 Count: XXX,XXX

Difference as % of 9054 Count:

Sampling Method: (Interval or Randomized File)

Problems/Comments:

State:

New Status Determinations (Tax Pop 3)

Calendar Year: YYYY

Universe: XXX,XXX

581 Count: XXX,XXX

Difference as % of 581 Count:

Sampling Method: (Interval or Randomized File)

Problems/Comments:

State:

Successor Status Determinations (Tax Pop 3)

Calendar Year: YYYY

Universe: XXX,XXX

581 Count: XXX,XXX

Difference as % of 581 Count:

Sampling Method: (Interval or Randomized File)

Problems/Comments:

State:

Terminations/Inactivation Status Determinations (Tax Pop 3)

Calendar Year: YYYY

Universe: XXX,XXX

581 Count: XXX,XXX

Difference as % of 581 Count:

Sampling Method: (Interval or Randomized File)

Problems/Comments:

State:

Field Audit Quality (Tax Population 5)

3 Quarters End: 09:YYYY

Universe: XXX,XXX

581 Count: XXX,XXX

Difference as % of 581 Count:

Sampling Method: (Interval or Randomized File)

Problems/Comments:

U.S. Department of Labor	Employment and Training Administration
	http://www.doleta.gov/regions/