Sage Journals: Discover world-class research

Abstract

Background

The Evidence-based Policy-Making Act of 2018 requires that Federal agencies use their data to develop statistical evidence to support policy and programmatic decisions.

Objective

This study assessed Vocational Rehabilitation agency capacity to effectively use their data to inform evidence-based decision-making.

Methods

The Capacity Survey assessed agency capacity in data management, data visualization and statistical analysis. The survey asked for details about (1) the availability of relevant software programs (e.g., SPSS for statistical analysis), and (2) staff expertise to use that software.

Results

Results pointed to capacity gaps that would significantly hinder most agencies’ application of even a simplified return on investment model. When examining statistical capacity, 60% of agencies responded “not applicable – staff do not have competence in the listed software packages (including SPSS, SAS, R, Python, Stata, and other)” and 71% of respondents said they lacked internal statistical capacity to analyze data using any of the listed programs.

Conclusions

Results suggested that many state VR agencies lack internal capacity to meet requirements outlined in the Evidence-Based Policy-Making Act of 2018 or regulations related to reporting and performance accountability requirements. Potential solutions to overcome capacity deficits include expanding internal capacity, expanding agency/consultant partnerships, and building cross-agency collaborations.

Keywords

vocational rehabilitation evidence-based decision making return on investment

Get full access to this article

View all access options for this article.

References

Anderson

C. A.

Sabella

Groomes

Tansey

Homa

Zhu

Connelly

Walker

(2021). The importance of building capacity for program evaluation and quality assurance within vocational rehabilitation. Rehabilitation Counselors and Educators Journal, 10(2). https://doi.org/10.52017/001c.29113

Balcazar

Taylor-Ritzler

Garcia

(Nov, 2023). Vocational Rehabilitation counselors’ perspectives on rehabilitation success. Journal of Applied Rehabilitation Counseling. https://doi.org/10.1891/JARC-2023-0013

Bipartisan Policy Center . (n.d.). Frequently asked questions related to the commission on evidence-based policymaking’s report. Bipartisan Policy Center. Retrieved December 20, 2023, from https://bipartisanpolicy.org/download/?file=/wp-content/uploads/2019/03/CEP-FAQs.pdf

Bishop

Lee

Zhou

Iwanaga

Chan

Tansey

(2022). Identifying vocational rehabilitation outreach and service training priorities: A national survey from diverse perspectives. Journal of Vocational Rehabilitation, 56(3). https://doi.org/10.3233/JVR-221188

Capella

M. E.

Andrew

J. D.

(2004). The relationship between job satisfaction and consumer satisfaction in vocational rehabilitation. Rehabilitation Counseling Bulletin, 47(2), 205–214. https://doi.org/10.1177/0034355204040040201

CIO.gov . (n.d.). Policies & Priorities: Foundations for Evidence-Based Policymaking Act of 2018. CIO.Gov. Retrieved December 6, 2023, from https://www.cio.gov/policies-and-priorities/evidence-based-policymaking/

Clapp

C. M.

Pepper

J. V.

Schmidt

R. M.

Stern

S. N.

(2023). The Effects of Vocational Rehabilitation for People Who are Blind or Vision-Impaired (SSRN Scholarly Paper No. 4514543). Social Science Research Network. https://doi.org/10.2139/ssrn.4514543

Commission on Evidence-Based Policymaking . (2017). The Promise of Evidence-Based Policymaking: Report of the Commission on Evidence-Based Policymaking (p. 129). https://bipartisanpolicy.org/download/?file=/wp-content/uploads/2019/03/Full-Report-The-Promise-of-Evidence-Based-Policymaking-Report-of-the-Comission-on-Evidence-based-Policymaking.pdf

Dean

Pepper

Schmidt

Stern

(2019). The effects of youth transition programs on labor market outcomes of youth with disabilities. Economics of Education Review, 68(C), 68–88. https://doi.org/10.1016/j.econedurev.2018.11.006

10.

Dean

Pepper

J. V.

Schmidt

Stern

(2017). The effects of vocational rehabilitation services for people with mental illness. Journal of Human Resources, 52(3), 826–858. https://doi.org/10.3368/jhr.52.3.0114-6111R1

11.

Dean

Pepper

J. V.

Schmidt

R. M.

Stern

S. N.

(2015). The effects of vocational rehabilitation for people with cognitive impairments. International Economic Review, 56(2), 399–426. https://doi.org/10.1111/iere.12109

12.

Dean

Schmidt

Pepper

Stern

(2018). The effects of vocational rehabilitation for people with physical disabilities. Journal of Human Capital, 12(1), 1–37. https://doi.org/10.1086/696098

13.

Eckstein

Sevak

Wright

(2017). Personal characteristics of vocational rehabilitation applicants: Findings from the survey of disability and employment. Journal of Vocational Rehabilitation, 46(2), 177–185. https://doi.org/10.3233/JVR-160853

14.

Foley

Boeltzig

(2011, April 13). Findings from the vocational rehabilitation research and training center. In Presentation to the 2011 council of state administrators of vocational rehabilitation conference, Bethesda, MD. https://www.explorevr.org/content/findings-vocational-rehabilitation-research-and-training-center

15.

H.R.4174 – Foundations for Evidence-Based Policymaking Act of 2018, No. H.R.4174 . (2019). https://www.congress.gov/bill/115th-congress/house-bill/4174

16.

Leahy

M. J.

Chan

Lui

(2014a). Evidence-based best practices in the public vocational rehabilitation program that lead to employment outcomes. Journal of Vocational Rehabilitation, 41, 83–86. https://doi.org/10.3233/JVR-140704

17.

Leahy

M. J.

Chan

Lui

Rosenthal

Tansey

Wehman

Kundu

Dutta

Anderson

C. A.

Del Valle

Sherman

Menz

F. E.

(2014b). An analysis of evidence-based best practices in the public vocational rehabilitation program: Gaps, future directions, and recommended steps to move forward. Journal of Vocational Rehabilitation, 41(2), 147–163. https://doi.org/10.3233/JVR-140707

18.

Leonard

D’Allura

Horowitz

(1999). Factors associated with employment among persons who have a vision impairment: A follow-up of vocational placement referrals. Journal of Vocational Rehabilitation, 12(1), 33–43. https://doi.org/10.3233/JVR-1999-00006

19.

Orszag

P. R.

(2009). M-10-01 Memorandum for the heads of executive departments and agencies. (Memorandum No. M-10–01; p. 7). Office of Management and Budget. https://www.whitehouse.gov/wp-content/uploads/2018/10/m10-01.pdf

20.

Rehabilitation Services Administration . (n.d.-a). View Data | Rehabilitation Services Administration. Retrieved November 30, 2023, from https://rsa.ed.gov/data/view-data

21.

Rehabilitation Services Administration . (n.d.-b). VR Program Reference Guide | Rehabilitation Services Administration. Retrieved October 23, 2024, from https://rsa.ed.gov/about/programs/vocational-rehabilitation-state-grants/vr-program-reference-guide

22.

Sabella

S. A.

Markve

M. E.

Groomes

D. A.

Shoemaker

M. T.

(2018). Competencies for effective program evaluation and quality assurance in vocational rehabilitation. Journal of Rehabilitation, 84(3), 66–74.

23.

Schmidt

R. M.

Clapp

C. M.

Pepper

J. V.

Stern

S. N.

(2019). Applications of the VR-ROI Project: ROI estimates for Virginia and Maryland (SSRN Scholarly Paper No. 3532014). Social Science Research Network. https://papers.ssrn.com/abstract=3532014

24.

Shoemaker

M. T.

Sabella

S. A.

(2010). The summit group. Journal of Rehabilitation Administration, 34(2), 105–112.

25.

Tansey

Bishop

Iwanaga

Zhou

Chan

(2023). Vocational rehabilitation service delivery: Technical assistance needs of vocational rehabilitation professionals. Journal of Vocational Rehabilitation, 58(1), 49–62. https://doi.org/10.3233/JVR/221224 https://doi.org/10.3233/JVR-221224

26.

Taylor

(August, 2024). Vocational Rehabilitation service provider capacity survey results. Council of State Administrators of Vocational Rehabilitation & Vocational Rehabilitation Technical Assistance Center for Quality Management. https://gvs.georgia.gov/document/document/csavr-vrtac-qm-provider-capacity-survey-report-final-9-12-24docx/download

27.

US Government Accountability Office . (2005). Vocational Rehabilitation: Better Measures and Monitoring Could Improve the Performance of the VR Program | U.S. GAO. https://www.gao.gov/products/gao-05-865

28.

US Government Accountability Office . (2007, May). Vocational Rehabilitation: Improved information and practices may enhance state agency earnings outcomes for SSA beneficiaries. https://www.gao.gov/assets/gao-07-521.pdf

29.

US Government Accountability Office . (2014, November). Program Evaluation: Some Agencies Reported that Networking, Hiring, and Involving Program Staff Help Build Capacity. GAO. https://www.gao.gov/assets/gao-15-25.pdf

30.

U.S. House of Representatives . (2017). H. Rept. 115-411—Foundations for Evidence-Based Policymaking Act of 2017 (2017/2018) [House Report]. https://www.congress.gov/congressional-report/115th-congress/house-report/411/1

Assessing Vocational Rehabilitation Agency Capacity to Engage in Evidence-Based Decision Making

Abstract

Background

Objective

Methods

Results

Conclusions

Keywords

Get full access to this article

References