Frequently Asked Questions on Current Performance Initiatives

Common performance measures are a management tool. The value of implementing a set of common performance measures is the ability to describe in a similar manner the core purposes of the workforce system—how many people found jobs; did they keep their jobs; and what were their earnings. Key attributes include:

Universal language for measuring performance.
Employment-focused measures for adult programs, and skill attainment measures for youth programs.
Designed in partnership with other federal employment and training agencies. (August 2005)

---------------------------------------------------------------------

A.2 Question:

Why do we need common measures?

Answer:

The common performance measures are an integral part of ETA’s performance accountability system, and ETA will continue to collect from grantees the data on program activities, participants, and outcomes that are necessary for program management and to convey full and accurate information on performance of workforce programs to policymakers and stakeholders. We see four key benefits from common measures:

Focus on the core purposes of the workforce system; employment for adults and skill attainment for youth
Break down barriers to integration resulting from different definitions, data and reports for each workforce program.
Resolve questions raised by GAO and other oversight agencies regarding consistency and reliability of data.
Reduce confusion among our customers and stakeholders who want to know about results. (August 2005)

---------------------------------------------------------------------

A.3 Question:

How will the common measures be implemented by the Department of Labor?

Answer:

ETA will collect outcome information on the common performance measures through revised reporting requirements. http://www.doleta.gov/Performance/reporting/. In other words, through the revised reporting requirements, grantees will report the data necessary to calculate the common measures.
For revisions to reporting instructions for WIA Title 1B, Wagner-Peyser/VETS and TAA programs, visit: http://www.doleta.gov/Performance/guidance/wia.cfm (August 2005)

---------------------------------------------------------------------

A.4 Question:

How will those currently enrolled in WIA, Wagner-Peyser/VETS and TAA programs be transitioned into the new Common Measures?

Answer:

Individuals currently enrolled in these programs do not become part of the common measure outcomes until they have exited.

For WIA Title 1B programs, the participant and exit cohorts that are required to be included on the November 2005 quarterly report for WIA adults and youth can be found in ETA’s revised WIA reporting requirements at http://www.doleta.gov/Performance/guidance/WIA/WIA-Quarterly-Report-Specifications-060505.doc

Wagner-Peyser/VETS: The participant and exit cohorts that are required to be included in the November 2005 quarterly report for Wagner-Peyser/VETS can be found in the ET Handbook No. 406 at http://www.doleta.gov/Performance/guidance/wia.cfm.

TAA: Because the TAPR is a report on exiters, the new data collection for Trade, beginning with the submission of the February, 2006 quarterly, will reflect common measures. The TAPR instructions are available at http://www.doleta.gov/Performance/guidance/wia.cfm. (August 2005)

---------------------------------------------------------------------

A.5 Question:

Will the common measures be the only information collected on ETA programs?

Answer:

No. Measurement of performance, and management and oversight of programs will continue to require the collection of information that is relevant and important to each discrete program. Further, some statutes explicitly state what information must be collected, and the common measures do not supercede those statutory requirements. ETA will continue to collect all the data on program activities, participants, and outcomes that are necessary to convey full and accurate information on the performance of workforce programs. The collection of information beyond that required for common measures will also help frame and provide context to the outcomes reported through these measures. For example, states will continue to report information on certain employer services, collected through the 9002 E report, so the Department can adequately convey information on utilization and other metrics that are not incorporated within the common measures. (August 2005)

---------------------------------------------------------------------

A.6 Question:

Under what youth measures will WIA Youth Programs report outcomes for PY2005?

Answer:

States are required to report against the seven core measures for youth (four older youth and three younger youth measures) as specified in section 136 of the Workforce Investment Act and as defined by TEGL 7-99. In addition, states are required to report against the youth common measures for placement in employment or education and attainment of a degree or certificate as specified in TEGL 28-04. States are also encouraged, but not required, to begin reporting against the literacy and numeracy gains measure for those youth who enroll on or after July 1, 2005. Youth who enroll prior to July 1, 2005 will not be reported in the literacy and numeracy gains measure even if they continue services beyond July 1, 2005. All states will be required to report outcomes for the literacy and numeracy gains measure beginning July 1, 2006 for those youth who enroll on or after July 1, 2006. (August 2005)

---------------------------------------------------------------------

A.7 Question:

Will States have to report the efficiency measure?

Answer:

No. Although program efficiency was identified as a common measure for Federal job training and employment programs in TEGL 15-03, ETA has revised its policy and will not require states and grantees to report on this measure. Instead, ETA will use existing program management data to report program efficiency at a national level to OMB. (August 2005)

---------------------------------------------------------------------

Section B: PROGRAM PARTICIPATION AND EXIT

B.1 Question:

Who is a program participant?

Answer:

The term “participant” is defined as an individual who is determined eligible to participate in the program and receives a service funded by the program in a physical location (e.g., a One-Stop career center). States and grantees are encouraged to consider as participants those individuals who receive services that are available electronically and are not accessed through a physical location.

Operational Parameters :

The criteria that are used to determine whether an individual is eligible to participate will be based on the guidelines for the program.
The term “service” does not include a determination of eligibility to participate in the program, self-directed job search that does not result in a referral to a job, services and activities specifically provided as follow-up services or regular contact with the participant or employer to only obtain information regarding his/her employment status, educational progress, need for additional services, or income support payments (except for trade readjustment allowances and other needs-related payments funded through the TAA program or National Emergency Grants (NEGs)).
Individuals who visit a physical location for reasons other than its intended purpose (e.g., use of restrooms or asking staff for directions) are not participants.

General Discussion : Many states have already developed data management systems capable of tracking core self-service and informational activities that individuals access remotely by collecting personally identifiable data and other basic contact information. ETA encourages all states to develop such capabilities so that sufficient program management data will be available to provide the Administration, Congress, and our stakeholders with a more complete and accurate picture on the levels of participation and types of services being provided through the nation's workforce investment system.

Discussion for Wagner-Peyser/VETS : Wagner-Peyser Act services are available to all customers (universal access); therefore no formal determination of eligibility is required. DVOP/LVER services are available to veterans, eligible persons and transitioning service members (TSMs) who meet the applicable eligibility criteria. A description of the eligibility criteria associated with the new homeless veteran reporting element is provided in Question I.6 below. (August 2005)

---------------------------------------------------------------------

B.2 Question:

What is the minimum amount of data that needs to be collected from a self-service participant?

Answer:

At a minimum , states should request that self-service participants provide their Social Security Numbers (SSNs) and employment status (employed or not employed) at participation date.

Upon request of the SSN, the customer must also be offered the opportunity to provide Equal Employment/demographic information. Per 29 CFR Part 37, customers must be offered the opportunity to self-identify the following characteristics: race/ethnicity, gender, date of birth (age) and disability status. Individuals should be made aware of the reason for the request of such information as well as the parties to whom disclosure may be made. (August 2005)

---------------------------------------------------------------------

B.3 Question:

Can an individual be a participant in more than one program?

Answer:

Yes and, in fact, that is the essence of the One-Stop service delivery model. Individuals can be a participant in several different programs, either sequentially or concurrently based on their specific needs and service plans. An individual is a participant in every program that funds the services received by the individual. When an individual is a participant in more than one One-Stop partner program, that individual is “co-enrolled.” Clearly, co-enrolling participants is a means to leverage limited resources while expanding service options.

When providing individual records for WIA and TAA programs (the WIASRD and TAPR, respectively), states must use the same unique identification number for an individual who is served by more than one program. In particular, unique identifiers contained with each TAPR file must be the same as unique identifiers contained within the Workforce Investment Act Standardized Record Data (WIASRD) files, referring to exactly the same person in both systems. Co-enrollment must be reported in each file.

The outcomes for individuals who are in more than one program are attributed to each program in which the individual is served. (August 2005)

---------------------------------------------------------------------

B.4 Question:

Are individuals who access services through the Internet considered to be participants?

Answer:

States and grantees are encouraged to consider as participants those individuals who receive services that are available electronically and are not accessed through a physical location.

---------------------------------------------------------------------

B.5 Question:

Please clarify the exclusion of “self-directed job search that does not result in a referral to a job” from the definition of a service.

Answer:

TEGL 28-04 (Common Measures Policy) indicates that “self-directed job search that does not result in a referral to a job” is not considered a service. This exclusion is limited to the act of conducting a job search, job bank browsing or modifying job search requirements (e.g., salary limits). There are some self-directed job search activities that are not included in this definition and should be considered as a reportable service. Examples of such activities include, but are not limited to, building resumes, accessing workforce information, and enrolling in an online job search mini-courses. (August 2005)

---------------------------------------------------------------------

B.6 Question:

Can an individual be counted as a participant more than once during a program year?

Answer:

Yes. Under the revised reporting requirements, a participant will exit after 90 days of not receiving services. If the job seeker comes back into the system (e.g. they exit the system and then return for additional service beyond the 90 day window), the job seeker is then considered a new participant.

A job seeker can be counted as a participant multiple times during the program year. Outcome data must be tracked for each participant's record; thus, the same individual can be “counted” in performance calculations more than once. (August 2005)

---------------------------------------------------------------------

B.7 Question:

Is there a difference between WIA and Wagner-Peyser / VETS definitions for “Recently Separated Veteran”?

Answer:

Yes. For WIA, the definition of a recently separated veteran is specified in the WIA statute as an individual who applied for participation under WIA Title I programs within 48 months after discharge or release from active duty (Section 101(49)(B). For Wagner-Peyser /VETS , a recently separated veteran is specified in Title 38 as someone who is within 36 months of discharge or release from active duty. (August 2005)

---------------------------------------------------------------------

B.8 Question:

What is the definition of program exit?

Answer:

The term “program exit” means a participant does not receive a service funded by the program or funded by a partner program for 90 consecutive calendar days and is not scheduled for future services.

Operational Parameters :

The term “service” does not include a determination of eligibility to participate in the program, self-directed job search that does not result in a referral to a job, services and activities specifically provided as follow-up services or regular contact with the participant or employer to only obtain information regarding his/her employment status, educational progress, need for additional services or income support payments (except for trade readjustment allowances and other needs-related payments funded through the TAA program or NEGs).
Grantees that use services provided by partner programs to extend the point of exit must have the capacity to track program participants until the individual exits all services funded by the program or the partner program.
The phrase “and is not scheduled for future services” does not apply to participants who voluntarily withdraw or drop out from the program. In these circumstances, once a participant has not received any services funded by the program or a partner program for 90 consecutive calendar days and has no planned gap in service, the date of exit is applied retroactively to the last day on which the individual received a service funded by the program or a partner program.

General Discussion : To encourage service integration and recognize shared contributions toward performance outcomes, workforce programs can share accountability under the common performance measures when the participant has exited all services funded by the program or funded by a partner program. Some methods for tracking participants across programs include specifying services financially assisted by partner programs in the individuals' service plans, coordinating service tracking through integrated data systems or other technologies, and providing coordinated follow-up services to individuals. (August 2005)

---------------------------------------------------------------------

B.9 Question:

Will there always be a common exit date if states use services provided by partner programs to extend the point of exit?

Answer:

It depends. Having a common exit date depends upon the state's ability to track partner services. If a state has the capability of tracking partner services, then the point of exit will occur based on the last service funded by any of the partner programs that the state can track. If a state does not have the ability to track any partner service, then the point of exit will occur on the last date of service funded by the specific program. (August 2005)
---------------------------------------------------------------------

Section C: REPORTING SCHEDULE

C.1 Question:

Will States continue to report using the rolling-four quarter methodology?

Answer:

YES. States will provide reports using the rolling-four quarter methodology. The November 14, 2005 quarterly report will therefore contain four quarters of data for participants (total count) and exiters (total count of exiters plus the exit-based performance measures of entered employment rate, employment retention rate, and six months earnings increase).

Discussion for Wagner-Peyser/VETS : For Wagner-Peyser/VETS, a complete reporting schedule for PY 2005, PY 2006, and PY 2007 can be found in the ET Handbook No. 406, which is available at http://www.doleta.gov/Performance/guidance/wia/Employment-Training-406-Handbook-060505.pdf. In order for the report due 11/14/05 to represent four quarters of services and outcomes, states are expected to “reach back” and re-calculate the number of participants, the number of exiters, and outcome data for entered employment, retention and earnings increase back to 7/1/03 . Because of time lags inherent to reporting and wage-records cross-match, each outcome measure is dependent on a different participant and exiter cohort.

Discussion for WIA Adults and Dislocated Workers : For WIA Adults and Dislocated Workers, a complete reporting schedule for the Annual and Quarterly Reports can be found in the reporting instructions, which are available at http://www.doleta.gov/Performance/guidance/wia.cfm.

Discussion for TAA : The reporting instructions for the TAA program requires states to submit individual exiter records via the Trade Act Participant Report. The rolling-four quarter methodology is not applicable to submission of these records. (August 2005)

---------------------------------------------------------------------

Section D: DATA SOURCES

D.1 Question:

Will supplemental sources of information continue to be collected and reported by states and grantees?

Answer:

YES. Supplemental data will be used for program management purposes and to gain a full understanding of program performance and activities. Although a majority of employment situations will be covered by wage records, certain other types of employment, particularly self-employment, are either excluded from the sources of data identified under Section 6 of TEGL 28-04 (Common Measures Policy) or very difficult for grantees to access due to data confidentiality issues (e.g., access to State Department of Revenue or Tax records). States and grantees should not be discouraged from providing entrepreneurial training or assisting the hard-to-serve simply because the subsequent employment is not covered by wage records. Therefore, in order to convey full and accurate information on the employment impact of ETA programs, grantees may use supplemental sources of data to document a participant's entry and retention in employment. For grantees that do not have access to wage records, supplemental sources of data will be permitted as an interim means of reporting on the earnings increase measure until all grantees in a program have access to wage records.

Allowable sources of supplemental information for tracking employment-related outcomes in the performance measurement periods include case management notes, automated Wagner-Peyser /VETS system administrative records, surveys of participants, and contacts with employers. All supplemental data and methods must be documented and are subject to audit.

Discussion for Wagner-Peyser/VETS : While UI wage records will serve as the primary data source for tracking outcomes for Wagner-Peyser/VETS, states may utilize other automated data sources, including the Wage Record Interchange System (WRIS), the U.S. Postal Service, the U.S. Department of Defense, state and local government employment records, etc., to document and obtain credit for employment and retention measures. Supplemental data, such as case management notes or employer verification of date of hire, are also allowable data sources for documenting employment and retention for workers not covered by UI wage records. States should, however, weigh the benefit of establishing potentially expensive follow-up systems for customers who take advantage of core Wagner-Peyser/VETS services. (August 2005)

---------------------------------------------------------------------

D.2 Question:

Will supplemental sources of information be allowed in the calculation of outcomes?

Answer:

Yes, see chart below, which was taken from TEGL 28-04.

PERFORMANCE MEASURE	DATA SOURCE
Adult Measures
Entered Employment	Wage records and supplemental data sources
Retention	Wage records and supplemental data sources
Earnings Increase	Wage records Supplemental data sources ( only for grantees that do not have access to wage records)
Youth Measures
Placement in Employment or Education	Wage records and supplemental data sources for placement in employment and military Administrative records for placement in education or training
Attainment of a Degree or Certificate	Administrative records
Literacy and Numeracy Gains	Assessment instrument

Agreements are not currently in place to provide access to wage records for grantees operating the following programs: H-1B Technical Skills Training, Job Corps, Migrant and Seasonal Farmworkers, Native American Employment and Training, Responsible Reintegration of Youthful Offenders, and Senior Community Service Employment Program. For these grantees, supplemental sources of data, such as participant surveys or contacts with employers, will be permitted as an interim means of reporting until all grantees in a program have access to wage records.

Discussion for Wagner-Peyser/VETS : While UI wage records will serve as the primary data source for tracking outcomes for Wagner-Peyser/VETS, states may utilize other automated data sources, including the Wage Record Interchange System (WRIS), the U.S. Postal Service, the U.S. Department of Defense, state and local government employment records, etc. Supplemental data, such as case management notes or employer verification of date of hire are also allowable data sources for documenting employment and retention for workers not covered by UI wage records. States should, however, weigh the benefit of establishing potentially expensive follow-up systems for customers who take advantage of core Wagner-Peyser/VETS services. (August 2005)

---------------------------------------------------------------------

D.3 Question:

How can states access wage record information on Federal and military employment?

Answer:

ETA has been sponsoring a pilot project since November 2003 through the State of Maryland to obtain access to these records. Data matches for DOD, OPM and USPS all became accessible in 2004. For more information, visit
http://www.doletAnswer:gov/performance/FEDES-Pilot-Project-Factsheet-021405.doc (August 2005)

---------------------------------------------------------------------

D.4 Question:

TEGL 28-04 references that states and grantees may exclude earnings related to severance pay or other earnings attributable to termination from an employment situation when calculating results for the adult earnings increase measure. It goes on to say that such a determination by the grantee must be based on documented procedures and processes for editing and cleaning wage record data and is subject to audit. How can states determine the amount of wages that actually reflects “severance pay” when using wage records and/or WRIS?

Answer:

To find out more about determining severance pay, contact states participating in the Integrated Performance Information Project. (The website for information on this project is www.wtb.wa.gov .) Given that states have access to historical quarterly wage data on each individual, it may be possible to see whether severance has been paid out in a given quarter. Statistical analysis of substantial “spikes” in the quarterly wage history of the individual or verification of an individual's quarterly earnings with the employer would be two ways to identify a potential severance payout. (August 2005)

---------------------------------------------------------------------

Section E: YOUTH COMMON MEASURES

E.1 Question:

What is the definition of "other organized programs of study"? Are there any specific examples of what programs would NOT meet the definition?

Answer:

“Other organized programs of study” is intended to be broad and include all formal education and training programs. DOL does not have examples of programs that would not fit the definition. (August 2005)
---------------------------------------------------------------------

E.2 Question:

Would any organization that is on a state Eligible Training Provider list be acknowledged as meeting the definition of an education provider or "other organized program of study"?

Answer:

We have not defined "other organized program of study" but it is reasonable to include Eligible Training Provider programs as falling into this category. (August 2005)

---------------------------------------------------------------------

E.3 Question:

What differentiates occupational and technical skills training from "generic pre-employment or work readiness skills" training?

Answer:

Occupational and technical training relate to a specific occupation or industry, whereas pre-employment or work readiness training are more general and do not relate to a specific occupation or industry. (August 2005)

---------------------------------------------------------------------

E.4 Question:

Do we need to go back and assess out-of-school youth who will be carried over into PY 2005 or will these youth be excluded from the literacy and numeracy gains performance measure?

Answer:

No back entry is necessary. Only new youth who enroll in the program on or after July 1, 2005 will be tracked under the literacy/numeracy gains measure. During PY05, states are being encouraged to begin data collection and reporting for literacy/numeracy gains measure but are not required to do so. All states will be required to begin reporting performance outcomes on the literacy/numeracy gains measure beginning in PY06. (See Page 4 of TEGL 27-04, which can be accessed at: http://wdr.doleta.gov/directives/attach/TEGL27-04.pdf ) (August 2005)

---------------------------------------------------------------------

E.5 Question:

If a youth participant remained basic skills deficient after the first year and continued in the program for an additional 340 days into the 2nd year, would this youth be excluded from the measure in the second year since they did not complete a full second year?

Answer:

Yes, this youth is excluded. TEGL 28-04 states that i ndividuals who remain basic skills deficient and continue to participate after completing a full year in the program, including pre-test and post-test, must continue to receive basic skills remediation services. These participants must be included in the measure for the first year of participation. They are not included again in the measure until they have completed a second full year in the program. At the completion of the second year, these participants must be post-tested and included in the measure. To determine an increase of one or more levels, the participant's post-test scores from the second year in the program must be compared to the scores from the test that was administered at the latest point during the first year. (August 2005)

---------------------------------------------------------------------

E.6 Question:

If the same youth discussed in the question above had post-testing done during the second year but again left before completing the second full year in the program, would this youth be included or excluded in the second year measures?

Answer:

Excluded. This person would not be counted in the second year measures because they had not completed a second full year in the program. (August 2005)
---------------------------------------------------------------------

E.7 Question:

Will any out-of-school youth be exempt from testing based on their education status (e.g., high school graduate, attending post-secondary education)?

Answer:

No. (August 2005)

---------------------------------------------------------------------

E.8 Question:

For the literacy/numeracy gains measures, what is the process for adding additional tests that do not appear on Attachment D of the Common Measures TEGL?

Answer:

States can propose tests they want to use to ETA, and ETA will request the Department of Education review the test to determine if it is appropriate to add to the list of acceptable tests. If the Department of Education does not approve the test, it will not be allowed. Proposals should be sent to ETAperforms@dol.gov. (August 2005)

---------------------------------------------------------------------

E.9 Question:

The AMES test was previously listed in TEGL 15-03 as an acceptable test for use in the literacy/numeracy gains measure. TEGL 28-04 does not include this test; can it still be used?

Answer:

No. The Department of Education removed it from the list because it is an outdated test. Therefore, it's not one that would be approved for use per the process described in question E8 (August 2005)

---------------------------------------------------------------------

E.10 Question:

How will youth who receive adult services be counted in the measures?

Answer:

In the case of a youth participant who becomes a participant receiving adult services (i.e., receiving services funded with both youth and adult funds), the individual will be counted in both the youth and adult common measures. If a young person is a participant in the adult program, only the adult common measures apply. That being said, the exit-based measures are applied once the individual exits the program (or any other partner program). (August 2005)

---------------------------------------------------------------------

E.11 Question:

TEGL 28-04 changes the inclusion of participants still in secondary school at exit of ETA program. In the original TEGL 15-03, this population was excluded because DOL did not want to penalize programs for serving these youth. Why was this change made?

Answer:

The intent of this change was to ensure that youth served by the WIA program successfully achieve a high school diploma or equivalent as research indicates greater success in the workforce for those with diplomas than those without.

---------------------------------------------------------------------

E.12 Question:

Answer:

---------------------------------------------------------------------

E.13 Question:

Does the certificate definition include certificates awarded by private career schools licensed by a state?

Answer:

Setting a high standard and using a tight definition is necessary for a meaningful and consistent outcome measure and to assess the workforce system's ability to provide individuals with the credentials needed to succeed in the labor market. The definition is intended to ensure that certificates are:

• Developed or endorsed by employers;
• Tied to specific technical/occupational skills; and,
• Approved or awarded by an appropriate educational, governmental or employer entity.

Some of the certificates offered by state-licensed private career schools will not meet the criteria outlined in the certificate definition. This issue generally affects schools that have not been accredited by accrediting agencies recognized by the Department of Education. (August 2005)

---------------------------------------------------------------------

E.14 Question:

When determining eligibility, youth must be determined low-income and have a barrier. A commonly used barrier is basic skills deficiency. Under Common Measures, in-school youth are not bound by the Literacy / Numeracy measure and therefore will have no testing administered. When determining basic skills deficiency for eligibility purposes, can states use results from tests administered by the school and/or other tests that are not on the list of tests permitted in the literacy/numeracy measure?

Answer:

LWIAs can use a test administered by the school to determine basic skills deficiency status for in-school youth. Additionally, LWIAs may use any assessment they deem as valid for testing basic skills levels for in-school youth since the literacy/numeracy measure does not apply to in-school youth. (August 2005)

---------------------------------------------------------------------

E.15 Question:

How will gains in literacy and numeracy skills be measured?

Answer:

To maintain consistency with the implementation of the common measures by the Department of Education (ED), ETA is adopting outcome measure of educational gain outlined in the National Reporting System (NRS). The NRS was developed by ED's Division of Adult Education and Literacy for implementation of an accountability system for federally-funded adult education programs under WIA. As outlined in the NRS, there are two sets of educational functioning levels - six levels for Adult Basic Education (ABE) and six levels for English-as-a-second language (ESL) students. Each ABE and ESL level describes a set of skills and competencies that students entering at that level demonstrate in the areas of reading, writing, numeracy, speaking, listening, functional, and workplace skills. These descriptors provide guidelines for placing participants in educational functioning levels, based on performance on standardized tests.

To achieve a positive outcome for the measure, a participant who was basic skills deficient must demonstrate through post-test that he/she has advanced one or more educational functioning levels beyond the level in which he/she was initially placed at pre-test. (August 2005)

---------------------------------------------------------------------

E.16 Question:

Is there a waiver for counting youth with disabilities in the literacy and numeracy gains measure?

Answer:

No. Youth with disabilities are not excluded from the literacy and numeracy gains measure. When administering assessment tests, individuals with disabilities should be accommodated according to:

1. Section 188 of WIA;
2. Guidelines associated with the assessment test; or
3. State law or policy.

The Department of Labor's Office of Disability Employment Policy and ETA feel strongly that individuals with disabilities should be included in the common measures. It is important that individuals with disabilities experience full program participation, including tracking their progress in attaining literacy and numeracy skills. (August 2005)

---------------------------------------------------------------------

E.17 Question:

Do Individual Educational Plan (IEP) diplomas and other alternative secondary school exit documents awarded to youth with disabilities count as a degree in the attainment of a degree or certificate measure?

Answer:

Please check with your state education agency. Only diplomas that are accepted by the state education agency as equivalent to a high school diploma count toward the degree or certificate measure. Therefore, if a state accepts IEP diplomas or other alternative diplomas as equivalent to a high school diploma, those diplomas would count toward the degree or certificate measure. (August 2005)

---------------------------------------------------------------------

Section F: ADULT COMMON MEASURES

F.1 Question:

When we count back for pre-wages for the Adult measure, will we be looking at the quarters “prior to participation” even though this measure applies to dislocated workers?

Answer:

Yes. (August 2005)

---------------------------------------------------------------------

F.2 Question:

Adult Entered Employment-- If a participant is exited in the July – September quarter, when do we count them in the entered employment rate? In other words, what is the first quarter after exit?

Answer:

The first quarter after exit in this case is the October-December calendar quarter immediately following the July-September quarter. That is the quarter in which to look for employment status for the entered employment rate. (August 2005)

---------------------------------------------------------------------

F.3 Question:

Six months earnings increase: What quarters are used to calculate the pre-participation earnings (i.e., what are the second and third quarters prior to participation) and the post program exit earnings??

Answer:

Quarter 1

Quarter 2

Quarter 3

Quarter 4

Year1

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sept

Oct

Nov

Dec

Year2

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sept

Oct

Nov

Dec

Year3

Jan

Feb

Mar

Apr

May

Jun

Jul

Aug

Sept

Oct

Nov

Dec

Let's say the individual becomes a participant in Quarter 3 of Year 2 and exits during Quarter 4 of Year 2. The second quarter of earnings prior to participation is Quarter 1, Year 2 and the third quarter of earnings prior to participation is Quarter 4, Year 1 (see italics). The second and third quarters after the exit quarter are Quarters 2 and 3 of Year 3 (see underlined). (August 2005)

---------------------------------------------------------------------

F.4 Question:

Please clarify the earnings increase measure.

Answer:

Similar to the retention measure, the earnings increase measure applies to all those who were employed in the first quarter following exit from participation. Another way of saying this is the Retention and Earnings measures uses the same “cohort” to calculate performance – specifically, those employed in the first quarter after exit. This is verified through wage records only. Also note that, as long as an individual is employed in the first quarter after exit, they are included, regardless of whether or not they were included in the entry to employment measure. All participants – regardless of their labor force participation status at start of participation – are included in the Earnings Increase measure. (August 2005)

---------------------------------------------------------------------

Section G: WIA REPORTING

G.1 Question:

Regarding the new WIASRD Element #333 “Received Workforce Information Services,” what types of information count as “Workforce Information?”

Answer:

Workforce information includes information on state and local labor market conditions; industries, occupations and characteristics of the workforce; area business identified skills needs; employer wage and benefit trends; short- and long-term industry and occupational projections; worker supply and demand; and job vacancies survey results. Workforce information also includes local employment dynamics information such as workforce availability; business turnover rates; job creation; and job identification of high growth and high demand industries. (See also question I.3 under Wagner-Peyser/VETS reporting.) (August 2005)

---------------------------------------------------------------------

G.2 Question:

"Type of Employment Match" in field number # 71, 73 and 75 and "Wages in the Quarter" in field # 78, 79 and 80 are referred,

1 = UI Wage Records (In-State & WRIS)
2 = Federal Employment Records (OPM, USPS)
3 = Military Employment Records (DOD)
4 = Other Administrative Wage Records
5 = Supplemental through case management, participant survey, and/or verification with the employer
6 = Information not yet available
What coding values should be used when an individual is found to have earned wages through more than one type data source, e.g. UI wage records, and DOD wage records?

Answer:

If the individual is found in more than one source of employment, record the data source for which the individual's earnings are greatest. For example, if a participant's quarterly earnings were made up of $1,000 in UI wage records and $2,000 in DOD wage records, then the coding value would be 3 = Military Employment Records. (August 2005)

---------------------------------------------------------------------

G.3 Question:

Regarding i tem 603 - Occupational Code- are states required to report the occupation of the individual at the time of exit or in the quarter after the exit quarter?

Answer:

The revised WIASRD does not change from the existing WIASRD. States should continue their current process. (August 2005)

---------------------------------------------------------------------

Section H: NEG REPORTING

H.1 Question:

If a state has an NEG for health care, which provides funds to extend medical insurance for the participants, should the state report that individual on the WIASRD?

Answer:

No. HCTC NEGs are excluded from WIASRD reporting since these grants have their own forms for reporting, have no participants and are therefore not appropriate for WIASRD reporting. (August 2005)

---------------------------------------------------------------------

Section I: WAGNER-PEYSER/VETS REPORTING

I.1 Question:

How should states report participants who received Wagner-Peyser /VETS funded core employment and information service via the internet from a remote site?

Answer:

Many states have reporting systems that enable them to track job seekers who receive core or workforce information services remotely. The Department encourages all states to develop this capacity so that sufficient program management data will be available to provide business, our investors, the Congress and the general public a more complete and accurate picture of the level of services provided through the workforce investment system. (August 2005)

---------------------------------------------------------------------

I.2 Question:

What global exclusions apply to the Wagner-Peyser / VETS funded programs?

Answer:

Global WIA Outcome exclusions are new to the employment service. The global exclusions that apply are: institutionalization, health/medical or family care; reservists called to active duty (includes members of National Guard units as well as military reserve units); and deceased. Definitions for these terms can be found in the common measures TEGL as well as the ET 406 Handbook. (August 2005)

---------------------------------------------------------------------

I.3 Question:

Please clarify the new data element “Received Workforce Information.”

Answer:

The Department is exploring ways to determine the efficacy and impact of critical investments in core workforce information products and services. Workforce information includes, but is not limited to, information on state and local labor market conditions, local employment dynamics information, industries, occupations and characteristics of the workforce, area business identified skills needs, employer wage and benefit trends, short and long term industry and occupational projections, worker supply and demand, and job vacancies survey results. Workforce information may be accessed in person through self-help in the resource rooms, individual counseling or career guidance in a One-Stop Career Centers, or remotely via the Internet, and/or in ancillary agency sites.

Beginning July 1, 2005 on the ETA 9002 A, B, C and D reports, states will need to report data on participant receipt of workforce information as self or staff assisted services and sub-aggregate outcome data based on the receipt of workforce information. On the VETS 200 A, B and C reports, states will need to collect data only on participant receipt of workforce information as a staff-assisted service. These data will be used to gather baseline information to observe whether the outcomes for Wagner-Peyser/VETS participants who receive workforce information services occur at a higher rate than for Wagner-Peyser/VETS participants who did not access workforce information services. (August 2005)

---------------------------------------------------------------------

I.4 Question:

How should states report job openings on America's Job Bank (AJB) and State Job Banks?

Answer:

Many states download job openings from AJB into their state job bank. ETA encourages this practice to provide job seekers with access to additional employment opportunities to which they may not ordinarily be exposed. However, to provide for consistency in information reported on job openings, job openings initially listed with AJB and imported into the state job bank should be excluded from the count of jobs that states report on the ETA 9002 E Report. Job openings posted with AJB are reported separately. In fact AJB reports differentiate between jobs listing entered by businesses from those uploaded from state job banks. (August 2005)

---------------------------------------------------------------------

I.5 Question:

Is the definition of the term “Intensive Services,” as included in the revised version of the ET Handbook 406, the same definition as the one applied to the delivery of WIA Title 1B services?

Answer:

No. The term “Intensive Services,” as applied by the ET Handbook No. 406 to the reporting of Wagner-Peyser/VETS services provided by DVOP specialists and LVER staff (on ETA 9002 B and VETS 200 A, B, C reports), has a somewhat different meaning from that applied to the delivery of WIA Title 1B services. In response to a change in the authorizing legislation for veterans' employment services, the term “Intensive Services” has been added to the ET Handbook No. 406 to take the place of “Case Management,” which is a service that has been delivered primarily to veterans by DVOP and LVER staff members. The same two prerequisites that have applied previously to the provision of case management (conduct of an assessment and development of an Individual Employment Plan) will continue to apply to the delivery of “Intensive Services.”

In the context of services provided through the DVOP and LVER programs, the term "Intensive Services" is intended to apply to staff-assisted services that can be provided by DVOP specialists and LVER staff. Therefore, the delivery of “Intensive Services” within the context of Wagner-Peyser/VETS employment services is not intended to include basic skills training or other training services which are described in WIA section 134(d)(3) . Similarly, within the WIA service delivery context, there is a formal requirement that participants are to receive “core services” prior to receiving “intensive services.” Although a corresponding formal requirement does not exist within the Wagner-Peyser/VETS context, it is expected that participating veterans will receive employment services equivalent to “core services” prior to receiving “Intensive Services.”

For the ETA 9002 B report and the VETS A-C reports due 11/14/05, states are expected to determine whether or not services rendered during 10/01/04-9/30/05 period are staff-assisted intensive services, using the “reach-back” approach described earlier. (August 2005)

---------------------------------------------------------------------

I.6 Question:

Please clarify the new data element for Homeless Veterans.

Answer:

One element of the recent efforts to reduce homelessness has been the implementation of the Homeless Veterans' Reintegration Program (HVRP). Beginning July 1, 2005 , data on homeless veterans who have been provided employment assistance by DVOP and LVER staff will be reported on the VETS 200 C report (not the VETS 200 A or B reports). It is intended that this report will include information on those Homeless Veterans' Reintegration Program (HVRP) participants who are served by DVOP or LVER staff, as well as information on any other homeless veterans who are not HVRP participants but who are served by DVOP or LVER staff.

The community-based and faith-based grantees responsible for implementing HVRP are strongly encouraged to coordinate their efforts with the One-Stop Career Centers in their localities, specifically with the DVOP staff assigned to those facilities. The homeless veteran eligibility criterion, which governs eligibility for HVRP services, differs slightly from the eligibility criterion governing eligibility to receive services from DVOP/LVER staff. Therefore, in isolated instances, a veteran who is eligible for HVRP and other homeless services may not meet the DVOP/LVER eligibility criterion. In situations of this type, it is expected that the DVOP will make arrangements with appropriate staff to provide the required employment services to these homeless veterans.

Note : In the current version of the ET Handbook No. 406 (Expiration Date 9/30/05 ), Section II.C.5 titled Reporting of Homeless Veterans includes incorrect information about the relationship between the eligibility criterion for HVRP and the eligibility criterion for DVOP/LVER services. Therefore, the above information treating that topic is to be preferred over the corresponding information in the 406 Handbook. This inconsistency in the 406 Handbook will be corrected in the version to be published following completion of the 60-day comment period scheduled to close on September 12, 2005. (August 2005)

---------------------------------------------------------------------

I.7 Question:

Under common measures, the 9002 C report does not include reporting on customer satisfaction survey (CSS) results for job seekers and employers. Does this mean we no longer need to conduct these surveys and report the results after the June 2005 quarter? If we no longer need to continue with the CSS for Wagner-Peyser /VETS , do we continue surveying customers up to or after June 30 th ?

Answer:

It is correct that, effective with the implementation of common measures, Wagner-Peyser/VETS will no longer require the collection and reporting of customer satisfaction scores for job seekers and employers. The August 15, 2005 quarterly report will be the final quarterly report on which states will report customer satisfaction results for Wagner-Peyser/VETS customers. While we do not expect states to conduct surveys on customers who register on June 21, 2005 , as an example, it is presumed that states will have attained the required number of completed surveys by the end of the program year in order to fulfill reporting obligations through the end of PY 2004. (August 2005)

---------------------------------------------------------------------

Section J: NATIONAL FARMWORKER JOBS PROGRAM

J.1 Question:

Are participants who receive related assistance services only counted in the common measures for the National Farmworker Jobs Program (NFJP)?

Answer:

No. Only those participants receiving employment and training services through the NFJP count in the common measures. TEGL 28-04, issued April 15, 2005 , states that, in general, all participants are to be taken into account when computing performance under the common measures, but describes two scenarios where participants may not be included. The second such scenario states that “there may be circumstances in which certain participants may not be included in one or more of the common measures because they are specifically excluded from performance measurement in the underlying program (emphasis added).” Such is the case in the NFJP.

Participants who receive related assistance services only are primarily employed farmworkers who require supportive and/or emergency services (i.e., related assistance services) in order to maintain their employment in agricultural labor while traveling the migrant stream. Their level of participation in the NFJP, therefore, will be excluded from calculation of the common measures. (August 2005)

---------------------------------------------------------------------

Data Validation

	WIA
Question:	Will states be able to electronically submit their PY03 data validation results directly to the Department of Labor?
Answer:	Yes. Instructions on how to e-submit your summary and analytical reports can be found in Appendix C of the user guide, which provides detailed instructions on the e-submit capability in the Data Validation Version 3.0MR software. A copy of the guide is available for each program on ETA's Web site at http://www.doleta.gov/performance/. Click on the link for Reporting Guidance and Data Evaluation and go to the third bullet point, Data Validation and more. States are required to electronically submit data element validation (DEV) and/or Report Validation (RV) data electronically via the Internet to the Department. The Department will assign user IDs and passwords for state staff to e-submit results through the validation for each of the three programs. Any questions should be forwarded to Traci DiMartini at Dimartini.Traci@dol.gov, or by phone at 202-693-3698. (November 2004) ---------------------------------------------------------------------
Question:	Who is responsible for submitting the data validation results for each program?
Answer:	The states decide who will submit the results for WIA, ES and TAA. Some states assign one person for all three programs and other states have one person for each program. The states should inform their regional representative of the contact person(s) for each state. The National Office will assign user ids and passwords based on this information. It is the state's responsibility to communicate any changes to the contact list to the regions as soon as possible. (November 2004) ---------------------------------------------------------------------
Question:	What is the difference between WIA Version 3.0.19 and WIA Version 3.0MR?
Answer:	There is no difference. WIA Version 3.0.19 and Version 3.0MR are the same. (November 2004) ---------------------------------------------------------------------
Question:	What are the enhancements in WIA Version 3.0 and its maintenance release, version 3.0MR?
Answer:	For Version 3.0MR, the enhancements are as follows: The validation software calculates the ETA quarterly report using the new time periods identified in TEGL 14-03 change 1. Fixes were made to the skill attainment, diploma rate, and dislocated worker earnings replacement rate for Table O. The validation software can now upload the report validation results without the user needing to draw a sample for data element validation. The ability to export and then re-import sub-samples for data element validation has been fixed. Version 3.0 of the WIA Validation Software has the following improvements: The validation software calculates the ETA annual report using the new time periods identified in TEGL 14-03. The validation software calculates and prints a facsimile report for Table O. It also produces a file in the EIMS format to upload the Table O results to ETA. The number of elements required for data element validation has been reduced to focus on eligibility, performance, and service-related elements. Filtering capabilities have been incorporated into the source table and performance outcome groups. These capabilities allow users to display records that meet user-defined criteria. The skill attainment calculations have been refined. Additional edit checks have been incorporated into the validation software. Version 3.0 is backward compatible. Users can open and use databases that were created in versions 2.0, 2.1, or 2.2 without losing prior validation results. (November 2004) ---------------------------------------------------------------------
Question:	Will states be sanctioned based on the results of PY03 validation?
Answer:	No. The results of PY03 validation will be used to set error rates for PY04. ETA will inform states of the benchmarks set for PY04 and any actions that may be taken if a state has abnormally high error rates. (November 2004) ---------------------------------------------------------------------
Question:	Our state may not be able to complete data element validation by the February 1, 2005 deadline. Can we request an extension?
Answer:	If a state believes it can not meet the February 1, 2005 deadline to complete data element validation it must contact its regional representative no later than November 30, 2004. States are required to provide an explanation detailing why the deadline imposes a hardship. The regions will work with the National Office to determine whether an extension can be granted. (November 2004) ---------------------------------------------------------------------
Question:	There are several elements that allow the state to use an MIS system to validate their results. Can you please provide specific examples of acceptable MIS systems? Can a state use its own WIA system to validate elements?
Answer:	The data element validation instructions in the WIA Validation Handbook dated November 2004 allow states to use their MIS as a source if the MIS has specific, detailed information that is stored in the state's information system that supports the date on the validation worksheet. An indicator alone, such as a checkmark on a computer screen, is not acceptable source documentation. For example, state MIS is an acceptable source documentation for date of first training service. To be an acceptable source to validate date of first training service, in addition to the date of first training, the state MIS should have detailed information about the type of training and the organization that provided the training. (November 2004) ---------------------------------------------------------------------
Question:	What happens if a state cannot find a record that has been sampled for validation?
Answer:	It depends upon the cause of the problem. If an office has been hit by a disaster (flooding, fire) that destroyed the records for that office (which the state should know prior to drawing the sample for validation), the state should inform the regional office of the issue. With the region's approval, the state should create a new extract file that excludes the records for that office. This new file should be imported into a different database from the one in which the state performed report validation. The software will then provide the state with a sample of records that excludes the records from the office that was destroyed. If individual files are missing from an office, then the validator should check the Missing Records box on the validation worksheet. (November 2004) ---------------------------------------------------------------------
Question:	States create their extract files for validation in October, using the wage record files available. Sometimes, when they go to validate the wages, the wages in the UI wage record system have changed because they have been updated. How should states deal with the dynamic nature of wage records?
Answer:	Wage records are dynamic information: new information may arrive after the participant extract file is created. States need to account for this when validating wages. In other words, they must find a way to determine the actual wages at the time they calculate their annual report. This can be done in many ways, including freezing the wage record file by making a copy of the appropriate records at the time the extract file is created or working with the state's UI division to obtain the accurate wage record information. (November 2004) ---------------------------------------------------------------------
	LABOR EXCHANGE
Question:	Explain the validation process for the Labor Exchange.
Answer:	States build two validation files, one for job seekers and one for job openings, according to the record layout specifications in the software user guide. The job seeker file is imported into the software, and the software generates validation values for the 9002 A, B, C and D and VETS 200 A, B and C reports. If the state uses a system other than DART or the ETA provided reporting/validation software version 1.9, then the state must also import its reported counts for the 9002 A, B, C and D and VETS 200 A, B and C reports. The software then generates a report validation summary, which compares the reported and validation values and calculates a percent difference. The software also generates a 25 record data element validation sample of job seeker records, that the state must compare to source document to ensure that the state constructed its validation file correctly, that is, that it included the correct data elements in building the validation file. The job opening file is imported into the software, and the software generates validation values for the 9002 E report. If the states uses a system other than DART or the ETA provided reporting/validation software version 1.9, then the state must also import its reported counts for the 9002 E. The software then generates a report validation summary, which compares the reported and validation values and calculates a percent difference. There is no data element validation of the job opening records. By November 15, 2004, states that are required to conduct report validation should validate their 9002 and VETS 200 reports for 200401, which is the four quarter period ending 9/30/04. All states should conduct data element validation for this same reporting period. (November 2004) ---------------------------------------------------------------------
Question:	Will states be able to electronically submit their validation results directly to the Department of Labor?
Answer:	Yes. States are required to electronically submit data element validation (DEV) and, if applicable, report validation (RV) data electronically via the Internet to the Department. Instructions on how to e-submit your summary and analytical reports can be found in the user guide, which provides detailed instructions on the e-submit capability in version 1.9 of the ES reporting/validation software. The Department will assign user IDs and passwords for state staff to e-submit results through the validation for each of the three programs. This information will be electronically submitted to state users by Monday, November 8, 2004. (November 2004) ---------------------------------------------------------------------
Question:	Who is responsible for submitting the validation results for each program?
Answer:	The states decide who will submit the results for WIA, ES and TAA. Some states assign one person for all three programs and other states have one person for each program. The states should inform their regional representative of the contact person(s) for each state. The National Office will assign user ids and passwords based on this information. It is the state's responsibility to communicate any changes to the contact list to the regions as soon as possible. (November 2004) ---------------------------------------------------------------------
Question:	What source documents are required for ES data element validation?
Answer:	States should compare the data on the validation worksheets to data in the state's MIS. (November 2004) ---------------------------------------------------------------------
Question:	When will the next version of the labor exchange validation software be released?
Answer:	Version 1.9 of the reporting/validation software should be available by November 8th. (November 2004) ---------------------------------------------------------------------
Question:	What are the enhancements in the new software?
Answer:	The enhancements in version 1.9 are: Corrected the double-counting of job seekers who have a staff-assisted service in the 2nd through 4th quarter of one registration year and then subsequently re-register for another registration year. Elimination of decimals in report validation summaries to mirror facsimiles and allow for export. Addition of batch printing for all report validation summaries, facsimiles, and worksheets. Corrected the calculation of rows 3a and 4a in 9002E. Corrected the calculation of transitioning service member report items throughout the 9002 and VETS 200 reports, in accordance with recent ETA/VETS guidance. Refinement of data element validation sampling logic to only sample records from performance groups of job seekers ready to be measured for retention (groups 1,2,3,4). Corrected the number of quarters for which neutral outcome records are accepted by the software (this error had no impact on reported counts). Enhancement to edit worksheets to indicate when a record is complete. Addition of data validation comments entry screen. Addition of summary and analytical report that displays number of sample records where there is a value, number of errors and data validation comments. Addition of capability to upload report validation and summary and analytical reports to ETA's Data Reporting and Validation System (DRVS). Improved error writing function to improve import times. (November 2004) ---------------------------------------------------------------------
	TRADE ADJUSTMENT ASSISTANCE (TAA)
Question:	Is Trade required to perform report validation?
Answer:	Since states do not calculate and submit performance reports to ETA, states are not required to perform report validation. The validation software, however, provides states with the capability to analyze their performance. (November 2004) ---------------------------------------------------------------------
Question:	When will the new TAA software be released?
Answer:	We expect the TAA validation software to be available by mid-November 2004. (November 2004) ---------------------------------------------------------------------
Question:	What are the enhancements to the new TAA software?
Answer:	The new version of the TAA validation software, version 1.3, has the following enhancements: The number of elements that states must validate is reduced, as per TEGL 3-03 change 1 Attachment A The software provides states with the ability to e-submit their data validation summary and analytical reports to ETA. The software allows states to create new databases from within the software A find function has been added to the summary of validation worksheets so that validators can easily find specific worksheets. Completed worksheets are highlighted in purple, making it easier for validators to determine which worksheets have been completed and which have not. (November 2004) ---------------------------------------------------------------------
Question:	What happens if a state cannot find a record that has been sampled for validation?
Answer:	It depends upon the cause of the problem. If an office has been hit by a disaster (flooding, fire) that destroyed the records for that office (which the state should know prior to drawing the sample for validation), the state should inform the regional office of the issue. With the region's approval, the state should create a new extract file that excludes the records for that office, and import the file into the software. The software will then provide the state with a sample of records that excludes the records from the office that was destroyed. If individual files are missing from an office, then the validator should check the Missing Records box on the validation worksheet. (November 2004) ---------------------------------------------------------------------
Question:	Can states use supplemental data to calculate workforce investment performance under the Department of Labor's common measures? (June, 2004)
Answer:	No. The Employment and Training Administration's Training and Employment Guidance Letter (TEGL) 15-03 indicates that Unemployment Insurance (UI) and other wage records will be the only allowable data sources for calculating the employment ¿related common measures. To maintain the integrity of common performance measures, ETA must ensure that the same data sources are used by each state to calculate outcomes. Self-employment is not covered under state UI systems. If supplemental data sources were allowed, the performance outcomes would not be calculated consistently across Federal programs. ETA understands that the common performance measures provide only part of the information necessary to effectively oversee the workforce investment system. In order to convey full and accurate information on the performance ETA will continue to collect all data on program activities, participants and outcomes. States should continue to collect and report supplemental data on employment outcomes of individuals not covered by UI wage records, such as the self-employed. When possible by state law, information on individuals who are self-employed that is obtained through record sharing or automated matching of state tax records may be used to calculate the common measures. The common measures policy is intended to ensure the comparability of the measures on a national level, not to discourage self-employment or entrepreneurship. ETA understands that excluding supplemental sources of information, such as that on self employment, from the calculation of the common measures may cause reported performance outcomes to be somewhat lower than if these sources were used. However, by continuing to collect supplemental information, ETA and the states will be able to determine the precise impact this information has on the common measures outcomes, permitting the adjustment of performance expectations as appropriate and potentially avoiding the unintended consequences of the measurement system. ---------------------------------------------------------------------
Question:	When must states and grantees complete data validation?
Answer:	Data Validation Update: The validation softwares for WIA, LX, and TAA provide the capability to e-submit the validation results. TEGL 3-03 change 1 and TEGL 3-03 change 2 provide the due dates and submission procedeures for PY 2002 validation. Please feel free to contact your regional representative or Traci DiMartini at 202-693-3698 with any questions. (updated March 24, 2004) ---------------------------------------------------------------------
Question:	The skill attainment rate does not match your calculations.
Answer:	ETA recognizes that the guidance and specifications for the younger youth skill attainment measure are vague, which can lead two people to calculate the measure in two different ways. States will not be held responsible if their values do not match the validation values. National Office program staff may provide additional policy guidance. (March 2004) ---------------------------------------------------------------------
Question:	Are states that use SPRA or DART software to calculate WIA annual reports required to use the ETA Validation software to perform reports validation?
Answer:	Yes. In TEGL 3-03 states were advised that they were not required to use ETA’s software. States were advised that if they chose to use a different methodology for validation, they must to notify their regional office or national grant program office of their intent to use a methodology other than ETA’s and to provide information that would support that the chosen methodology met the current validation requirements. To date, no state has made such a request. (November 2004) ---------------------------------------------------------------------
Question:	Can states submit separate records in the WIASRD for individuals enrolled in more than one funding stream? How does this affect data validation?
Answer:	According to TEGL 14-00 Attachment E, change 1, "if an individual is served jointly by multiple WIA title I-B funding sources/programs (e.g., youth and adult funds), only one record should be submitted. However, all sections relevant to each funding source/program must be completed. If the individual is served independently by multiple funding sources or local areas, separate records may be submitted." For the state level report (Tables B through M), the Validation software rejects records that are duplicates. A duplicate is two or more records that have the same SSN/ID, date of exit, and funding stream (adult, dislocated worker, older youth, or younger youth). If two or more records match on all three of these variables, they are considered duplicates and rejected by the validation software. Thus, for participants who are served by multiple funding streams, states may use either consolidated records or separate records for each funding stream. States, however, should use consolidated records for participants who received services from one funding stream and two or more WIBs. Otherwise, the software will reject the records as duplicates. (November 2004) ---------------------------------------------------------------------
Question:	Is there a quick way to find a customer after the data validation is completed?
Answer:	Yes. The validation software provides a find function that allows users to find specific records. (November 2004) ---------------------------------------------------------------------
Question:	Fields 30 and 43, occupational code system, and Field 44, Employed in the Third Full Quarter after Exit, on the layout and validation instructions do not match the TAPR. Which record layout should I follow?
Answer:	The software uses the TAPR record layout. Consequently, there printed record layout is incorrect. Users should follow the format listed in the TAPR. (March 2004) ---------------------------------------------------------------------
Question:	When monitors go on site are they allowing staff to make changes to the documentation in the files?
Answer:	Validators/Monitors should not allow staff to make changes to the documentation in the case files during the validation review in order to pass a particular item. (March 2004) ---------------------------------------------------------------------
Question:	Who performs data validation, the state or the local area?
Answer:	Validation is the state’s responsibility, and should be performed by a person who is independent of the data collection process. (November 2004) ---------------------------------------------------------------------
Question:	How do States request a deadline extension?
Answer:	States should contact their regional representative to request an extension. (March 2004) ---------------------------------------------------------------------
Question:	How do States submit the data?
Answer:	We are working with SAIC/MPR now to develop the process for states to electronically submit the data validation findings by uploading the file to EIMS using standard EIMS procedures. We anticipate having this process in place mid-April. In the interim, when states have finalized the validation, they should send an e-mail message to their regional representative with a copy to Gail Eulenstein with a brief narrative about the validation outcomes. States may also export the report and data element validation results into PDF format, and including them in an e-mail. To PDF a report, choose the export option - the envelope in the upper left corner of the screen. Click on OK. Type in the filename and save the PDF file. Contact MPR if additional assistance is needed. (March 2004) If you have additional questions, call Gail Eulenstein or send an email to Eulenstein.gail@dol.gov. ---------------------------------------------------------------------

Question:	When and how must states and grantees submit the results of data validation to ETA?
Answer:	TEGL 3-03 change 1 provides deadlines for states to submit their validation results. (November 2004) ---------------------------------------------------------------------
Question:	What training and technical assistance will be given to states and grantees?
Answer:	ETA and Mathematica Policy Research (MPR) conducted regional training sessions for states for the WIA, Labor Exchange, and Trade Adjustment Assistance programs during summer 2003. In addition, several Web-based data validation seminars were presented to state staff in November and December 2003. Training for National Program grantees will be provided separately for each program as follows: MSFW in November 2004. On-going technical assistance is available to states and grantees from MPR. States and grantees are encouraged to contact MPR for assistance to avoid unnecessary delays and problems when implementing data validation. (November 2004) ---------------------------------------------------------------------
Question:	Will states and grantees receive additional funds to complete data validation?
Answer:	No. The requirement to perform validation derives from states' and grantees' responsibility to provide accurate information on program activities and outcomes to ETA. States and grantees are expected to provide resources for conducting validation from their administrative funds. ETA has taken a number of steps to minimize the costs of data validation, including developing software that states and grantees can use, providing on-going technical assistance, and using a sampling approach that minimizes the number of participant case files that must be reviewed. (January 2004) ---------------------------------------------------------------------
Question:	Will states and grantees be held accountable for performing validation and for the accuracy of data?
Answer:	States and grantees will be required to validate data annually. Failure to complete data validation and submit the results to ETA will be deemed a failure to report, and subject to corrective action or sanction, as appropriate. States and grantees will be held accountable for meeting acceptable levels for the accuracy of data, once these standards are established. States and grantees that fail to meet accuracy standards will receive technical assistance from ETA and develop and implement a corrective action plan. In addition, data that do not meet accuracy standards may keep states and grantees from being eligible for incentive grants. Significant or unresolved deviation from accuracy standards may be deemed a failure to report. (January 2004) ---------------------------------------------------------------------
Question:	When will standards for the accuracy of data be established?
Answer:	Accuracy standards will not be established until the third year of validation. Therefore, there will be no sanctions based on discrepancies during the first and second years of validation. The first year of data validation is intended to be a learning experience and to focus on identifying and resolving problems with reporting systems. Data collected in the second year will be analyzed, and based on this information, accuracy standards will be established prior to the third year of validation. (January 2004) ---------------------------------------------------------------------
Question:	How do states and grantees know what is the acceptable documentation for data elements?
Answer:	TEGL 3-03 change 1, Attachment A provides a list of acceptable source documentation for WIA, TAA, and MSFW. The data validation handbooks that have been developed for each program provide further clarification of the documentation requirements (November 2004) ---------------------------------------------------------------------
Question:	Are electronic records of documents acceptable documentation?
Answer:	Yes. Source documentation that has been scanned and is only available electronically is acceptable documentation for data elements. (January 2004)
Question:	Can states continue to use self-attestation for eligibility determination purposes?
Answer:	TEGL 3-03 change 1 Attachment A clearly identifies the documents that are acceptable for each data element being validated. Self-attestation is only an acceptable source where it is listed as in the TEGL. (November 2004) ---------------------------------------------------------------------
Question:	Does the review of the sampled participant case files have to be conducted on-site?
Answer:	Although highly recommended, the review of participant case files does not have to be conducted on-site. If states do plan to conduct the review at a central location, they should advise regional office staff of their procedures and obtain the regional staff’s approval. (January 2004) ---------------------------------------------------------------------
Question:	Will ETA conduct monitoring of states' and grantees' validation efforts?
Answer:	Yes. After data validation is fully implemented, a monitoring guide will be developed to ensure consistency among all states and programs in conducting the validation process. (January 2004) ---------------------------------------------------------------------
Question:	What validation software is currently available?
Answer:	Validation software is currently available for states to complete validation for WIA, Labor Exchange, and Trade Adjustment Assistance programs. Software for the Migrant and Seasonal Farmworkers program should be available in early December. Software for the Native American Employment and Training program and the Senior Community Service Employment Program is under development. (November 2004)

[Download Adobe Acrobat Reader] [PDF to HTML Conversion]