STATE OF WEST VIRGINIA

UPDATE OF THE
FULL PERFORMANCE EVALUATION OF THE

CHILD PROTECTIVE SERVICES




Timeliness of Child Abuse Investigations Continues to Improve, but Variations Among Counties Remain


OFFICE OF LEGISLATIVE AUDITOR
Performance Evaluation and Research Division
Building 1, Room W-314
State Capitol Complex

CHARLESTON, WEST VIRGINIA 25305
(304) 347-4890


Update:Timeliness of Child Abuse Investigations Continues to Improve But Variations Among Counties Remain

This is the third compliance review of Child Protective Services (CPS). The original report was completed in 1996 based on 1995 CPS case records. Compliance review updates sampled cases from calendar years 1996, 1997 and 1998, respectively. For this 1999 review, the Legislative Auditor's Office reviewed 384 CPS cases for 1998 from 12 counties. Figure 1 shows the agency's progress since the original report. For the most part, the improvements that were documented in past compliance reviews have continued. However, there remains significant variation among counties.


Figure 1 shows that most of the improvement has occurred over the years in the agency's reduction of cases without a record of face-to-face interviews and interviews that exceed 90 days. In the original report of 1995 cases, 37% of the cases had no record of interviews of alleged victims. This percentage was less than 1% in the 1998 case sample. The percentage of cases that had interviews over 90 days was 11% in 1995 cases, compared to zero in 1998 cases.

Despite these improvements, the percentage of cases that had interviews between 15 and 90 days has been virtually constant at close to 20% in the last three years reviewed. Furthermore, several counties in the 1998 sample did not obtain the 14-day standard in 30% of their cases. Table 1 shows the average number of days it took to conducted interviews with alleged victims in cases where the interviews were beyond 14 days. In 1995, nearly 30% of the cases had face-to-face contact with alleged victims well after 14 days. The average time in these cases was 117 days. This has dropped substantially over the last three years. For 1998 cases, the average is 25 days. However, there were several cases that took between 40 to 76 days to finally interview alleged victims. On an individual basis, Mercer county took an average of 34 days when it exceeded the 14-day criteria. Although the risk of children being further abused has declined over the last three years because of improved response time, there still is reason to be concerned.



Table 1
The Average Time of Interviews When They Exceed 14 Days
1995 Cases1996 Cases1997 Cases1998 Cases
117 Days32 Days37 Days25 Days
Source: From samples of CPS cases.

Individual Results of 1998 Cases

The 1998 sample included seven counties that were reviewed in the 1997 case sample. These seven counties were selected for this audit update because they had the lowest response time in either CPS or APS in the 1997 sample. These seven counties are Jackson, Mercer, Monongalia, Preston, Putnam, Summers, and Taylor. In addition to these seven counties, five new counties were systematically selected that have never been reviewed by the the Legislative Auditor's Office. The counties are Brooke, Harrison, Mingo, Raleigh, and Randolph. The purpose for reviewing these new counties is to determine if the recommendations of the original audit are being implemented statewide as opposed to only the counties that were reviewed in previous reviews. Tables 2 and 3 provide a comparison of the counties with the lowest response time in the 1997 audit with the results of the 1998 sample.




Table 2
1997 CPS Cases: Counties With Lowest Response Time

County


Interviews Within 14 Days
Interviews Between 15 an 90 Days
Interviews over 90 Days
Cases Without Face-to-Face Interviews
Jackson*75.0%25.0%0.0%0.0%
Mercer57.1%42.9%0.0%0.0%
Monongalia*87.1%12.9%0.0%0.0%
Preston 44.4%44.4%11.1%0.0%
Putnam66.7%29.2%0.0%4.2%
Summers*96.0%4.0%0.0%0.0%
Taylor27.3%31.8%18.2%22.7%
* Selected to be evaluated based on their low response times in the 1997 APS sample.

As Tables 2 and 3 indicate, the counties reviewed last year showed significant improvement or maintained good performance the following year. The only exception was Mercer county which had a relatively low response time in both years. The greatest improvement occurred in Putnam, Taylor and Preston counties.

Table 3
1998 CPS Cases


County

Interviews Within 14 Days
Interviews Between 15 an 90 Days
Interviews over 90 Days
Cases Without Face-to-Face Interviews
Jackson87.1%12.9%0.0%0.0%
Mercer65.7%31.4%0.0%2.9%
Monongalia91.2%8.8%0.0%0.0%
Preston 83.5%16.7%0.0%0.0%
Putnam93.9%6.1%0.0%0.0%
Summers88.5%11.5%0.0%0.0%
Taylor82.6%17.4%0.0%0.0%


Review of Five New Counties Is Mixed

Table 4 contains the results of the sample of 1998 cases for Brooke, Harrison, Mingo, Raleigh, and Randolph counties. For Mingo county, over 90% of the cases were within 14 days of the referral. Harrison county had interviews within 14 days of referrals in nearly 85% of its cases. However, Brooke, Raleigh, and Randolph counties conducted face-to-face interviews beyond 14 days in nearly a third of their cases.

Table 4
1998 CPS Cases: Response Time of New Counties

County


Interviews Within 14 Days
Interviews Between 15 an 90 Days
Interviews over 90 Days
Cases Without Face-to-Face Interviews
Brooke69.0%31.0%0.0%0.0%
Harrison84.8%15.2%0.0%0.0%
Mingo91.2%8.8%0.0%0.0%
Raleigh68.6%31.4%0.0%0.0%
Randolph70.0%30.0%0.0%0.0%

Slow Response Times In Some Counties

In general, the results from the sample of 1998 cases show improvement in response times for several counties. However, four counties (Brooke, Mercer, Raleigh and Randolph) had difficulties meeting the 14-day time standard in nearly a third of their cases. This has been true for Mercer county in the sample of 1997 and 1998 cases. Table 5 shows the average number of days these four counties took to conduct face-to-face interviews when they exceeded the 14-day time standard.



Table 5
The Average Time of Interviews When They Exceeded 14 Days
1998 Cases
BrookeMercerRaleighRandolph
25 Days34 Days28 Days26 Days
Source: From a sample of 1998 CPS cases.

Staffing Problems Continue to Hinder CPS

Of the four counties that had relatively slow response times in the sample of 1998 cases, Mercer and Randolph identified the lack of staff as part of the problem. The Community Service Manager for Mercer county wrote the following in response to his county's 1998 performance:

During the period of time (1998) of this review the Mercer District was short CPS Intake staff. We had just hired a new CPS worker the end of November 1997. As it took several months to train her in all aspects of CPS we could assign few referrals to her. This left us short the first half of 1998. In addition, we lost a CPS worker in August 1998 and did not get to replace her until November leaving us short the second half of 1998.

The Community Service Manager for Randolph county wrote the following in response to her county's performance:

In regard to CPS, the statistics show that when the Randolph/Tucker District is fully staffed, we are only at 51% of case load standards which were determined Pre-FACTS....and FACTS has only served to be an added clerical burden to my staff. As you can see from page 1 of the chart, under Factors, we were often short at least one worker, which would mean we were operating at 44% of case load standards. As a matter of fact, by September, 1998, we had 4 of 7 workers in CPS with less than 1 year of experience.

Table 6 shows the number of allocated CPS positions compared to the number of filled positions. Eight of the twelve counties in the sample had at least one vacant position, of which five had two or more unfilled positions. Although Putnam county is listed as having no vacant positions, it had two vacancies during 1998 before they were filled. For several counties to have one or more vacancies only hinders the agency's ability to provide effective child protection.

Although vacant CPS positions may impact the agency's ability to be effective, it does not appear that it is a primary factor. A correlation analysis was conducted on the number of vacant positions for each local office and the local office's rate of compliance with the 14-day standard. The correlation coefficient equaled -0.202. The sign was appropriate, indicating that compliance suffers when vacancies increase. However, the correlation is relatively low. This suggests that there are other factors that influence the rate of compliance for local offices. These other factors include management controls, staff turnover, travel time needed to contact children, etc.


Table 6
Allocated & Filled Positions
Child Protective Services
CPS Positions
CountyAllocatedFilledVacant
Brooke871
Harrison1293
Jackson550
Mercer15132
Mingo862
Monongalia107.52.5
Preston440
Putnam770
Raleigh17152
Randolph761
Summers220
Taylor981
Allocated and filled positions as of December 31, 1998.


Investigation Process Continues to Improve

The review of 1998 cases found that the agency has increase the use of the Child at Risk Field System (CARF). Part of this system includes an Initial Assessment instrument (CPS-2). This instrument is important because it assists the agency in making objective conclusions as to what action should be taken by DHHR in a case. The review of 1995 cases in the original report indicated that only 56% of the cases used the Initial Assessment instrument. The sample of 1998 cases shows approximately 98% of the cases had the Initial Assessment instrument (see Table 7).


Table 7
County Performance Regarding the Completion of Initial Assessments

County
Percentage of Cases With Initial Assessments
County
Percentage of Cases With Initial Assessments
Brooke0.966Preston*0.900
Harrison1.000Randolph1.000
Monongalia1.000Taylor*0.913
Jackson1.000Mercer1.000
Mingo1.000Raleigh*0.971
Putnam1.000Summers1.000
* These counties are in the Family Options Initiative pilot project. They follow a different process that in some cases may not require the use of the Initial Assessment instrument.



Timely Responses Are Also A Concern Among Family Options Initiative Counties

Over the three compliance reviews, the Legislative Auditor's Office has reviewed four of the five counties that are part of the Family Options Initiative (FOI) pilot project. The five counties in the project are: Barbour, Fayette, Preston, Raleigh, and Taylor. Fayette county has never been reviewed by the Legislative Auditor. The FOI project was implemented in 1995 as a method of providing multiple levels of CPS intervention and services to all children and families that are reported to the agency through child abuse allegations. The Department of Health and Human Services describes the project as follows:

A case is either tracked into A, B, or C at the point of intake. If it is tracked A, it is assigned to a CPS worker to complete a full initial assessment and safety analysis. If it is tracked B, it is assigned to a CPS worker for completion of a safety check. If it is tracked C, it is given directly to a Community Provider with whom a letter of agreement exists to complete a short term service needs assessment and no more than 60 days of service provision. Upon completion of the service provision by the Track C Community Provider, a report form and billing invoice are submitted to the local CPS Supervisor.

The FOI program is intended to provide services to more families through community providers, and improve workload management by identifying cases that do not need full initial assessments, which can be time consuming to perform. DHHR completed its final evaluation report on this pilot program in June 1998. The agency claims that the findings show an increase in the total numbers of families served; clients were satisfied with services provided; the recidivism rate was lower than expected; services were provided to families that would not have received such services under the traditional CPS system. DHHR has considered implementing this program statewide.

DHHR has budgeted nearly $900,000 a year for the past four years towards the FOI program.
Table 8 shows the budget and actual expenditures. The agency indicated that the budgeted amount is in addition to the CPS system that exists in the pilot counties. Personal Services and Benefits reflect additional staff that were hired, Current Expenses are for administrative costs such as office expenses, travel and training. The Services category are expenses for treatment to children, such as counseling, provided by community organizations.



Table 8
Family Options Initiative
Budget & Expenditures

Fiscal Year


Budget

Total
Expenditures
Personal Service & Benefits
Current Expenses


Services
1996$899,500$429,881$241,693$122,942n/a
1997$899,500$636,556$478,749$112,113$42,261
1998$899,500$640,441$473,492$124,765$42,184
1999*$899,500$803,444$509,939$202,384$91,120
*Expenditures for 1999 are projected.
Source: Department of Health and Human Resources, Bureau for Children and Families.


Despite funds for additional staff, it is clear from this review that one important outcome of the pilot project, a timely response to referrals, is not being achieved. This is illustrated in Table 9. Each FOI county reviewed for the first time had relatively slow response times. Although Preston and Taylor had significant improvement compared to the review of 1997 case records, their performance for 1997 cases was among the poorest reviewed that year. Those counties both identified having staffing problems that year. In addition, one-third of Raleigh county's 1998 cases did not meet the 14-day standard.





Table 9
Performance of Counties Involved in the Family Options Initiative
Percent of Cases In Compliance with the 14-day Standard
1996 Cases1997 Cases1998 Cases
Barbour54.5%
Preston44.4%83.5%
Raleigh68.6%
Taylor27.3%82.6%


The scope of this review is to evaluate the responsiveness of local offices to allegations of abuse, which included counties in the FOI project. The effectiveness of the FOI project was not examined by the Legislative Auditor. However, as part of its evaluation of the FOI project, DHHR should determine if any aspect of the project may have contributed to the slow response times in these counties. In addition, it is imperative for DHHR to stabilize the CPS system. Under the current CPS system, there still is a need for the agency to address pressing staffing needs and reducing the staff turnover rate. Despite improvements made over the past three years, some counties are still struggling to respond to allegations of child abuse within a timely manner. Furthermore, it only takes the loss of one worker within any county to reduce performance for counties that are currently doing well. Stabilizing the system requires a decision to be made on whether or not to implement all or parts of the FOI project statewide, assessing management controls, and staffing issues.


Conclusions

Although improvements have continued, the review of 1998 cases reveals that CPS performance still varies significantly by county. Counties which were reviewed a second time often fair better than those which are reviewed for the first time. This reinforces the need for a statutory mandate that the Department of Health and Human Resources conduct detailed performance evaluations at the county level.

Even with this continued improvement in the area of conducting face-to face meetings within the state-mandate of 14 days, nearly one in five referrals (18.5%) still does not meet this time frame. This should be a concern for obvious reasons; children are at risk of further abuse the longer it takes for intervention. It also becomes more difficult to substantiate an allegation of abuse the longer it takes to conduct a face to face interview. For example, physical evidence of abuse may heal.


Finally, DHHR needs to work at continuing the progress made, as well as stabilizing the CPS system which still remains vulnerable. This can occur only by continued monitoring of the system, providing for the proper staffing, and addressing the issue of reducing staff turnover rate. There is also the need to reach a final decision on the Family Options Initiative project so as to immediately begin the process of solidifying CPS performance.


Recommendation 1:

The Department of Health and Human Resources should continue monitoring county CPS offices to ensure continued improvement in performance statewide.


Recommendation 2:

The Legislature should consider amending state law to require the Department of Health and Human Resources to conduct detailed performance evaluations, to include, but not be limited to, the child protective services program of every county office, once every two years. This legislation should require the Department of Health and Human Resources to prepare a full report of its findings and include any proposals to rectify deficiencies.


Recommendation 3:

The Department of Health and Human Resources should evaluate the appropriateness of continuing or replicating the Family Options Initiative statewide in order to begin stabilizing CPS performance. The agency should devote its resources to address the pressing needs of the current CPS system by continued monitoring of county performance, providing adequate staffing and reducing the turnover rate.