• Several City departments have contracted with community-based organizations to provide workforce development services, because departments consider community-based organizations to have closer ties to the local communities and economically disadvantaged adults. In FY 2006-2007, City departments and the Private Industry Council awarded $15.3 million in contracts to 59 different community-based organizations, or approximately 52.6 percent of the estimated funding of $29.1 million for workforce development programs, providing a range of workforce development services, from basic education, job readiness, and life skills training to more focused job training preparing participants for permanent placement.
  
  
• Although community-based organizations receive more than one-half of all funding for the City's workforce development programs, City departments cannot determine if this funding is effective in placing the City's economically-disadvantaged residents into permanent jobs. City departments do not consistently measure or ensure the effectiveness of programs provided by community-based organizations, nor do they have a comprehensive plan to ensure that all aspects of adult workforce development (e.g., soft and hard skills development, subsidized jobs, access to work supports, etc.) are provided and linked together. Consequently, clients do not receive the full range and depth of training and employment services that are needed to maintain a permanent job and ultimately achieve self-sufficiency.
  
  
• The performance data collected by community-based organizations, which primarily includes program participation and job placement data, is limited and varies from program to program. City departments lack a common definition of successful outcomes for workforce development programs and method to measure (1) if programs providing services other than direct job placement eventually result in job placement, or (2) if job placements are permanent and financially self-sufficient. As a result, City departments and community-based organizations are unable to determine the overall effectiveness of the workforce development services in terms of providing their clients with long-term, financially self-sufficient employment.
  
  
• The City departments contracting with community-based organizations for workforce development services lack consistent oversight of community-based organizations and a formal process to identify organizations that fail to meet performance goals across several funding sources. Having sufficient, consistent, and on-going oversight of community-based organizations is necessary in order to ensure that community-based organizations are effectively performing their contractual obligations and to establish a system of accountability and standards for the City's workforce development efforts.

As part of the City's broader efforts to provide adult training and employment services to City residents with barriers to employment, several City departments have contracted with community-based organizations to implement the City's workforce development programs. City departments perceive that community-based organizations have close contacts with the community and economically disadvantaged residents lacking access to educational and training programs.

In theory, community-based organizations' connections to community residents may make them more effective in recruiting and assessing residents for workforce development programs, especially in low-income neighborhoods; integrating employment and training with social service and community development activities; and developing community-sensitive, public policy agendas for labor market reform. Although community-based organizations may possess this theoretical 93benefit of place,94 some studies have shown that community-based organizations do not necessarily meet performance requirements, especially the successful placement of clients in jobs providing financial self-sufficiency, and retention and advancement of those clients. Despite such findings, community-based organizations have continued to play a central role in providing adult training and employment services to San Francisco.

Overview of San Francisco's Adult Workforce Development Community-Based Organizations

Community-Based Organizations' Funding and Services

In FY 2006-2007, several City departments and the Private Industry Council, funded by the Federal Workforce Investment Act, contracted with 59 different community-based organizations to provide adult training and employment services, totaling $15.3 million, as shown in Table 3.1 Of these contracts, the Human Services Agency awarded $6.6 million, or 43.1 percent of the contract amounts; the Department of Public Health awarded $3.1 million, or 20.2 percent of the contract amounts; and the Private Industry Council awarded $2.2 million, or 14.3 percent of the contract amounts.

Table 3.1

Total Amount Provided by City Departments & Private Industry Council to Community-Based Organizations
for Adult Workforce Development
FY 2006-2007

Of the 59 different community-based organizations that received funding in FY 2006-2007, thirteen received funding from more than one City department, as indicated in Table 3.2 below.

Table 3.2

Community-Based Organizations That Received Funding From More Than One City Department in FY 2006-2007

Source: Survey data collected by the Department of Economic and Workforce Development and data collected by the Budget Analyst from city departments and community-based organizations.

The 59 community-based organizations included in the survey offered a mix of services:

• 17 provided assessment, career counseling and other support services.

• 50 provided pre-employment training (e.g., job-readiness and adult education);

• 52 provided skills training (e.g., customized, incumbent worker, and new worker

• 33 provided job placement;

• 14 provided transitional work, internships, and other work experience; and

• 10 provided other types of services such as behavioral health counseling.

Table 1 in the Appendix shows all of the community-based organizations that received funding from city departments and the Private Industry Council in FY 2006-2007, including the types of services they provided to adult populations.

Populations Served

San Francisco's workforce development programs are generally designed to serve specific populations or communities rather than provide skills training for specific occupations or business sectors . Most of the Human Services Agency's programs target cash aid recipients while other City departments target other low-income residents. Of the community-base organizations that received funding from City departments²:

• 15 organizations served refugee or immigrant populations,

• 8 organizations served Temporary Assistance for Needy Families (TANF) or aid recipients,

• 7 organizations served homeless adults, and

• 3 organizations served ex-offenders or incarcerated adults.

Most of the community-based organizations indicated serving 93Other94 adult population groups, including (a) young adults; (b) poor, low-income, or disadvantaged individuals; (c) lesbian, gay, bisexual, and transgender groups; (d) people with disabilities; (e) people with HIV/AIDS; (f) Visitacion Valley residents; and (g) at-risk young adults what have been affected by critical incidents. Table 2 in the Appendix shows the adult populations served by the community-based organizations in FY 2006-2007.

Monitoring, Measuring, and Evaluating Community-Based Organizations' Performance

City departments' ability to adequately monitor, measure, and evaluate the performance of community-based organizations in providing workforce development services to adult populations is inconsistent. The effectiveness of community-based organizations in preparing residents for and placing them in jobs leading to financial self-sufficiency can be assessed based on:

(a) the range and depth of the services currently provided;

(b) the performance measures in place indicating the relative success of clients in terms of job placement, retention, and advancement; and

(c) the systems in place to ensure sufficient oversight by and accountability to city departments that provide funding to community-based organizations.

Range and Depth of Services Provided

Because City workforce development programs generally target populations with significant or multiple barriers to employment, community-based organizations often provide basic job readiness and life skills services rather than specific job skill training. As discussed above, the 59 different adult workforce development community-based organizations that received funding from City departments and the Private Industry Council, provided a mix of job readiness, pre-employment, and job training services. As the survey results show, most of the services provided by these community-based organizations focus primarily on training and career counseling instead of direct recruitment or placement of individuals to jobs.

City departments generally set workforce development goals based on the requirements of their funding sources. Departments have not uniformly defined placement into permanent jobs providing financial self-sufficiency as a goal when planning workforce development programs or funding community-based organizations to provide services.

• The Human Services Agency, which provides the largest portion of funding to community-based organizations to provide workforce development services to adult populations, must meet Federal work participation mandates for cash aid recipients of at least 20 hours per week in allowable activities, although these requirements do not include placement into permanent jobs leading to financial self-sufficiency.

• The Mayor's Office of Community Development, which administers the Federal Community Development Block Grant, funds community-based organizations that provide workforce development services to individuals who have multiple barriers to employment or are not served by other City programs. According to the Mayor's Office of Community Development, job placement is not a goal for all community-based organization contracts. The Federal Housing and Urban Development Department, which oversees the Community Development Block Grants program, does not require placement into permanent jobs leading to financial self-sufficiency as a condition of funding. According to the Mayor's Office of Community Development, even if placement is not a goal for a specific contract, the contractor must demonstrate that it is linked to other employment services as well as placement. The proposal has to show that the contractor has a relationship with an agency that assists with placement.

• The Department of Public Health provides services to people living with HIV/AIDS based upon both the funding source and the Department's determination of the need, and pre-employment and employment services to mental health consumers. These contracts provide counseling, case management and other services but do not specifically target job training and placement.

Other City departments and agencies provide workforce development programs through community-based organizations based on specific department or agency requirements. The Redevelopment Agency, for example, must place redevelopment area residents into construction and other jobs resulting from development activities. The Port provides work experience through the California Conservation Corps for young adults.

In FY 2006-2007, the Private Industry Council awarded federal Workforce Investment Act grants to community-based organizations to provide basic educational and English skills, job training, and other associated workforce development services. The Workforce Investment Act requires reports on the use of these funds, including the percentage of participants who have completed a program, retention rates in unsubsidized employment, and wages. However, as noted below, most of the community-based organizations that received funding from the Private Industry Council did not meet their performance targets in FY 2006-2007.

Most of the community-based organizations provided a mix of services, generally targeted to language, basic skills, job readiness, and job skills training. Of 85 separate community-based organization contracts³, only 33 specifically included job placement services. Many of the contracts with community-based organizations targeted services to individuals who have significant skills development needs, including language and basic educational skills, or significant social issues, such as homelessness or mental health issues. Therefore, the services provided were intended to provide basic skills or job readiness training to prepare individuals for employment.

However, City departments that provide funding to these organizations lack a comprehensive and cohesive plan to ensure that pre-employment, life skills, job readiness and other services are eventually linked to job training and placement. Workforce Investment San Francisco, with the support of the Department of Economic and Workforce Development, should develop a cohesive workforce development plan that clearly defines the roles of all workforce development service providers (e.g., the departments themselves, community-based organizations, employers, educational institutions, private entities, etc.), ensuring that clients' employment and training needs are met and that, ultimately, clients are placed in permanent jobs providing financial self-sufficiency. Taking advantage of the strengths of each of the key player in the City's workforce development network (e.g., the community-based organizations' community outreach capabilities, the City departments' ability to link soft and hard skills, etc.) and having a plan that links all the service providers, as well as the services they provide, are essential in managing the complexities involved in workforce development.

Performance Measures

Most contracts with community-based organizations require some data collection and reporting, but the data collected by community-based organizations is limited and varies from program to program. In addition, the definitions of performance measures (e.g., placement) currently used by community-based organizations vary from one organization to another. Based on the survey data collected by the Department of Economic and Workforce Development, the most common performance measures collected by adult workforce development community-based organizations were pre-employment training and job placement. Performance indicators collected typically include the number of participants completing training and the number placed in jobs. In addition, based on the Budget Analyst's interviews of community-based organization staff, only a few community-based organizations actually monitor and collect information on their clients' job retention prospects (e.g., after six months of completing the workforce development program), whereas most community-based organizations focus primarily on collecting program participation/completion information.

Based on available participation/enrollment and placement data for community-based organizations, placement rates⁴ ranged from 7 percent to 100 percent, with 50 percent being the median placement rate. The community-based organizations with available enrollment and placement data⁵ received a total of $8,323,649 in workforce development funding from City departments in FY 2006-2007 and provided services to 10,530 participants, of whom 2,256 were placed into jobs. The average cost per placement, therefore, was $3,690. This data, however, is incomplete. The Redevelopment Agency was only able to provide complete data for FY 2005-2006 but not for FY 2006-2007, and the Mayor's Office of Community Development was only able to provide projections rather than actual results for FY 2006-2007.⁶ Further, these participation and placement data were self-reported by City departments and community-based organizations and participation and placement data were not available for several community-based organizations. Table 3 in the Appendix shows all of the community-based organizations that have available program participation and/or job placement data.

While collecting program participation and job placement data is important, City departments do not require community-based organizations to consistently collect data to measure the effectiveness of the services they provide in terms of successful placement of clients in jobs that provide financial self-sufficiency, as well as retention and advancement of those clients. The performance measures currently collected by the surveyed community-based organizations do not include job retention rates or information on job upgrades or wage increases during the follow-up. Such limitations in the performance data and the data collection practices of community-based organizations make it difficult to measure overall progress of workforce development program clients along a self-sufficiency continuum and ultimately assess the overall effectiveness of the City's network of workforce development initiatives and efforts.

These limitations and inconsistencies in performance measure data can be attributed to the nature of the services provided and/or the varying expectations and requirements of the community-based organizations' funding sources (e.g., City departments), as described in the examples below:

• Human Services Agency: Most of the community-based organizations funded by the Human Services Agency primarily collect data on the number of enrollees or program participation and placement of clients in unsubsidized employment. Based on the Human Services Agency's definition, placement simply refers to the client being placed in post-training employment (i.e., without any measures of placement duration, job advancement, or the client's self-sufficiency). Community-based organizations are required to submit monthly Attendance and Progress reports for all welfare-to-work participants, which are then reviewed by Human Services Agency case workers.

• Mayor's Office of Community Development: Community-based organizations funded by the Mayor's Office of Community Development provide a variety of services based on five-defined activities: soft skills training, hard skills training, job counseling, placement, and retention services. The Mayor's Office of Community Development does not track job placement data if this was not one of the community-based organization's activities, although the community-based organization may track placement data if job placement tracking is required by other funding sources. If job placement is not a goal, the Mayor's Office of Community Development measures other participation data.

Overall, City departments have no common definition of successful outcomes for workforce development programs provided by community-based organizations and no method to measure (1) if programs providing services other than direct job placement eventually result in job placement, or (2) if job placements are permanent and financially self-sufficient. Community-based organizations offering workforce development services aimed at self-sufficiency should track participants and measure their advancement and performance to monitor the overall effectiveness of the City's workforce development efforts. Workforce Investment San Francisco should coordinate programs provided by City departments through community-based organizations, establishing standard policies, processes and procedures for measuring performance across all community-based organizations, and tracking outcomes for clients served by the community-based organizations.

Oversight and Accountability

Although City departments have systems in place to monitor community-based organizations, the current systems of oversight are inconsistent and vary from one department to another.

Transfer of Contract Management to the Department of Economic and Workforce Development from the Private Industry Council

Only three of the community-based organizations receiving federal Workforce Investment formula funds from the Private Industry Council in FY 2006-2007 met their enrollment goals⁷. In 2007-2008, due to the dissolution of the Private Industry Council, the Department of Economic and Workforce Development will administer the federal Workforce Investment Act grant, including management of contracts with community-based organizations, and will enter into new agreements in FY 2007-2008 with these three community-based organizations: Arriba Juntos, Mission Language and Vocational School, and Self Help for the Elderly. Management of contracts with community based organizations providing workforce development services is a new function for the Department of Economic and Workforce Development, although the Department is developing policies and procedures to address weaknesses in the Private Industry Council's financial and contract management of the Workforce Investment Act grant.

In order to streamline contract administration, the Budget Analyst recommended during the June 2007 Budget and Finance Committee budget hearing that the Department of Economic and Workforce Development work with the Mayor's Office of Community Development, which administers the Department's General Fund contracts, to determine the feasibility of the Mayor's Office of Community Development administering contracts funded by the Workforce Investment Act grants as well, although this recommendation was not accepted by the Committee (Files 07-0739 and 07-0740).

Mayor's Office of Community Development and Human Services Agency Contract Oversight

In FY 2007-2008, in addition to contracts for workforce development services funded by the Community Development Block Grant, the Mayor's Office of Community Development will administer the Redevelopment Agency's three contracts with community-based organizations as well as the Department of Economic and Workforce Development's contracts with community-based organizations funded by the General Fund. According to the Mayor's Office of Community Development, contracts with community-based organizations are monitored monthly to determine whether the organization met its contractual goals each month through the on-line database. If the organization fails to meet its goals for three months, the contract manager can override the automatic system and withhold contract payment until performance goals are met. According to the Mayor's Office of Community Development, many but not all contractors receive at least one site visit per year.

The Human Services Agency conducts semi-annual site visits as needed for program start-up, participant file review, and quarterly progress review in order to evaluate the quality of the contracted services provided by community-based organizations and to ensure compliance with the contract. Community-based organizations are required to submit monthly attendance and progress reports for all Welfare-to-Work participants, which are then reviewed by Human Services Agency case workers.

The Controller's Joint Fiscal and Compliance Monitoring Project

In order to coordinate, streamline, and ensure adequate oversight of community-based organizations that receive funding from more that one City department, the Controller has initiated a project in which these community-based organizations receive one standard fiscal monitoring site visit or self assessment each year. At the beginning of the year, the participating departments meet to determine which community-based organizations require closer monitoring and site visits and which organizations only require self assessment. In FY 2006-2007, the community-based organizations providing workforce development services that received site visits were:

• Arriba Juntos,

• Catholic Charities,

• Conrad House,

• Episcopal Community Services of San Francisco,

• Glide Foundation,

• Jewish Family and Children's Services,

• Charity Cultural Services Center, and

• Ella Hill Hutch Community Center. ⁸

Self Help for the Elderly was the only community-based organization funded by more than one City department and providing workforce development services that required self assessment rather than a site visit.

The Controller's program only addresses community-based organizations' fiscal and compliance management but not other performance issues. Although City departments include performance goals and monitor performance in contracts with community-based organizations, and collaborate informally when funding the same community-based organization, City departments lack a more formal process to share information on contractor performance. Having sufficient, consistent, and on-going oversight of community-based organizations is necessary in order to ensure that community-based organizations are sufficiently and effectively performing their contractual obligations and to establish a system of accountability and standards for the City's workforce development efforts.

Community-Based Organizations: Some Lessons Learned

Although no single model exists for designing and implementing workforce development strategies and programs that work best for community-based organizations, a review of the workforce development literature indicates some lessons learned from past research and experiences. Below are just some of these lessons:

• Community-based organizations should collaborate with one another to learn and gain strength, find out what employers are looking for, and respond to these employers' needs. In most cases, no single organization can do it all and thus multiple organizations, agencies, and entities must work in coordination through partnerships and other arrangements.

• Many innovative initiatives combine elements from multiple workforce development models and are relatively comprehensive in the range of the services they provide.

• A community audit in the workforce development context identifies the needs, resources, and gaps in the local workforce development system. Conducting or participating in a community audit as part of the strategic planning process can help community-based organizations make informed decisions about how to change the way they do business and position themselves appropriately in the local workforce development system.

• If community-based organizations are to effectively assist people in moving from poverty to self-sufficiency, they will need to provide or establish linkages to more than basic education, short-term skills training, and job placement. Workers need services that support job retention, career management, and connections to post-secondary education.

• Community-based organizations that are involved in workforce development must be willing to adopt a dual-customer approach, in which they simultaneously serve employers and individual job seekers. Community-based organizations focusing their workforce development programs on helping individuals achieve self-sufficiency know that employers play a key role in worker retention and advancement.

• A community-based organization should have clear criteria for what constitutes a good job, such as placement in a high-demand occupation, certain wages, benefits, and access to education and training opportunities.

• Funding entities and community-based organizations should create a comprehensive and standardized performance measurement system to track their clients' specific progress in moving toward self-sufficiency. Performance measure data should be collected, entered, and analyzed by the organizations on a regular basis.

Community-based organizations, under contract with City departments, play an important role in providing community outreach and adult training and employment services to San Francisco residents, but community-based organizations provide a limited range of workforce development services and City departments lack a comprehensive plan to ensure that all aspects of adult workforce development are provided and are linked together. In addition, community-based organizations do not collect data on whether clients are successfully placed in permanent, decent-paying jobs, and City departments currently do not have a coherent definition of successful workforce development outcomes. Further, City departments lack consistency in their oversight of and accountability for community-based organizations and the workforce development services they provide. Such weaknesses in monitoring, measuring, and evaluating community-based organizations' performance limit the City departments' ability to determine the true impacts and value of these organizations in providing workforce development services to San Francisco residents.

Workforce Investment San Francisco, with the support of the Director of Workforce Development of the Department of Economic and Workforce Development, should :

Implementation of all recommendations should be accomplished using existing resources. The benefits include vastly improved monitoring, measuring, and evaluation of the effectiveness of workforce development services provided by contractual community-based organizations. In addition, establishing a full accounting of all funding provided to community-based organizations, as well as the services provided and populations served, would result in a reduced risk of duplication of efforts and resources for the City's workforce development initiatives. Furthermore, City departments would increase efficiency and reduce unnecessary costs through better reporting on performance results and improved oversight of community-based organizations' activities. Lastly, holding City departments and community-based organizations accountable for the results of their efforts would ensure that San Francisco residents are receiving workforce development services that would result in successful job placement and self-sufficiency.

1 In FY 2006-2007, the Human Services Agency provided funding to the Ella Hutch Community Center, the Northern California Service League, and the Young Community Developers via funding through the Private Industry Council.

2 This analysis of the types of populations served included only those community-based organizations that received funding from City departments, (i.e., excluding those that received funding from the Private Industry Council) because the Private Industry Council did not provide information on the populations served by its community-based organization grantees.

3 The 59 discrete community-based organizations held 85 separate contracts with nine City departments.

4 Placement rate was calculated by dividing the total number of people placed in jobs by the total number of program participants/enrollees.

5 This represents 32 different community-based organizations.

6 According to the Mayor's Office of Community Development, complete FY 2006-2007 data will be reported in September 2007.

7 According to the Department of Economic and Workforce Development, other community-based organizations received funding in FY 2006-2007 but were not included in the assessment because they were not candidates for funding in FY 2007-2008.

8 This Controller's list of community-based organizations that received funding from more than one City department differs from the Budget Analyst's list because the Budget Analyst's list focused only on those community-based organizations that provided workforce development services to adult populations in FY 2006-2007. Additionally, the Controller's list only includes community-based organizations that have contracts with participating departments.