OLA#: 021-03

LEGISLATIVE ANALYST REPORT

TO: Honorable Members of the Board of Supervisors

FROM: Anthony Ababon, Office of the Legislative Analyst

DATE: December 12, 2003

SUBJECT: CitiStat Technology Systems (File No. 031154)

SUMMARY OF REQUESTED ACTION

A motion (sponsored by Supervisor Newsom) requested that the Office of the Legislative Analyst (OLA) research CitiStat technology systems in Baltimore and review how other major cities including New York, Chicago, and Phoenix have implemented data management systems similar to CitiStat.

EXECUTIVE SUMMARY

Based on the CompStat crime reduction system deployed in New York City and other jurisdictions, CitiStat is a management strategy originated in Baltimore using technology to gather statistics and manage performance of city departments. CitiStat and other performance management systems are not strictly technology programs, they are methods of managing information and leveraging technology for the purpose of improving efficiency.

The OLA surveyed the implementation of CitiStat in Baltimore and similar systems and processes in Chicago, New York, and Phoenix. The systems were adopted because the cities were concerned with holding departments accountable for their performance. In addition, the cities wanted to track the departments' progress in meeting goals and objectives set by city officials and believed that the best way to track progress is to quantify measurements of department performance. (See Appendix for an example of indicators) Also, the cities wanted to improve customer service delivery to the public.

While data about the performance of departments is currently collected twice a year by the San Francisco Controller's Office, the Controller is not responsible for holding departments accountable for their performance-that responsibility falls to the Mayor and Board of Supervisors. Based on the experience of other cities, adopting a CitiStat approach would make the review of department performance and the identification of problems more frequent. The CitiStat approach would also shorten the time period to implement changes aimed at improving efficiency and productivity if combined with an enforcement mechanism or a system of financial incentives for holding departments accountable.

In the event that San Francisco decides to pursue a CitiStat-type implementation, strong executive support from the Mayor and Board of Supervisors is essential to the success of such an interdepartmental management project. Before adoption of a CitiStat program, a full assessment of the current data management systems and performance measurements of several City departments needs to be performed by an internal or external organization in order to study the feasibility of integrating them into a single database accessible to City officials. San Francisco may set up a dedicated CitiStat office with a designated city official to oversee the management process and analysis.

BACKGROUND

CitiStat is a statistics based approach to performance management in Baltimore. CitiStat represents a management approach based on holding department staff accountable for trends in selected indicators of department performance. CitiStat is not a standardized management system nor does it only consist of technology tools. Similar principals are applied with different forms and names depending on the jurisdiction. For the purposes of this report, CitiStat refers to the implementation in Baltimore.

A CitiStat system includes collecting specific, accurate, and real-time performance data from department databases that is sent to a common database. Department heads and representatives meet regularly with city officials at CitiStat meetings to discuss fluctuations in performance data and develop strategies to improve or sustain the performance of each participating department. CitiStat meetings-the forum for the presentation of performance data that occur bi-weekly, monthly or quarterly-enable department heads, staff, and city officials to review and analyze measurements of department performance and efficiency. A dedicated CitiStat team of performance data analysts comprised of new or existing trained city employees prepares the data of participating departments that is presented at CitiStat meetings.ⁱ

The process of performance management requires the identification of the proper performance measures to track for each department. Among other responsibilities, City officials, department management, and the team of data analysts identify and develop measures for performance, develop processes to track necessary information, and determine appropriate targets for performance. Technologically, measurements of department performance and efficiency require electronic database systems and information systems that are capable of delivering real-time data on performance indicators selected by the City.

The vision that accompanies requiring frequent and regularly scheduled meetings with department staff and city officials is a more responsive, cost-effective, and efficient City government. The anticipated outcomes of adopting the CitiStat approach to performance management are responding to constituent requests for city services in a more timely manner, reducing redundancy of programs, and reducing City employee absenteeism and over-time expenses. These outcomes were the most frequently mentioned of the cities studied, although cost-savings derived from implementing the approach are not guaranteed.

In addition, data and performance management systems are often integrated with centralized call centers (e.g. the "3-1-1" systems in Baltimore, Chicago, New York) through which constituents can request information, place requests for city services, and follow-up on the progress of their requests. The integration of a centralized call center with department databases enables the departments and the City to track its efficiency in responding to constituent requests.

Current Practice in San Francisco

Currently, both the Budget Analyst Office and the Office of the Controller conduct management audits of City departments and programs. The Controller's Performance Management and Audits divisions monitor the performance of the departments. The Audits division performs about two to three performance audits of selected departments per year.ⁱⁱThe Controller follows Generally Accepted Government Auditing Standards when it examines current processes, conducts interviews with management and staff, conducts surveys of other cities and counties for best practices, and evaluates management controls or tools used to guide and monitor operations. In addition, the Controller follows up six, twelve, and twenty-four months after reports are distributed and states that a vast majority of recommendations are implemented within two years.ⁱⁱⁱ

Using existing technology infrastructure, the Controller's Performance Management division updates department performance measures and data twice a year - early in the fiscal year on October 1 and mid-fiscal year in February as part of the annual budgeting process. In October, City departments report on performance for the prior fiscal year and any adjustments to their performance targets for the current year. Mid-fiscal year, departments submit performance data for the first six months, projected performance for the current fiscal year, and performance targets for the coming budget year. Each spring, the Controller also releases performance measures in the annual "City Survey" report.^iv

The Budget Analyst performs comprehensive management audits as assigned by the Board of Supervisors and follows professional standards that must be met in conducting audits. The Budget Analyst examines the entire department including analysis and evaluation of management practices, command structure, department reports and web presence, organizational structure, staffing, technology, and expenditures and revenues. Audit recommendations approved by the Board of Supervisors are enforced through the budget process.

Other methods presently used by a few departments to review departmental performance include the following:

San Francisco's www.sfbizinfo.com, coordinated by the Small Business Commission (SBC), is an example of a collaborative effort to streamline service and information delivery by several City agencies providing related services. As the "central point of information and referral" for the business community, "sfbizinfo" assists with business registration, license services, and permit information. The "sfbizinfo" system is accessible in written form, on the Internet, and through a centralized call center staffed by trained professionals.

The stated mission of SBC is to promote and maintain a healthy small business climate in San Francisco. As the coordinator for "sfbizinfo", SBC ensures that participating public and private agencies work collaboratively. SBC meets monthly with its "core team" of public and private employees and meets quarterly with "sfbizinfo" contacts from participating agencies to ensure that departments fulfill performance measures determined by SBC.

From any of the "sfbizinfo" channels, each customer request is entered into a centralized database that is accessible to SBC and tracked until closed. SBC prints out weekly performance reports from the centralized database enabling SBC to track the progress of requests and hold agencies accountable for fluctuations in performance.

With regard to technology, the San Francisco Department of Telecommunications and Information Services (DTIS) operates a robust private data network and a nationally recognized Geographic Information System (GIS) for mapping and location based analysis. DTIS also has experience with large-scale citywide system implementation. DTIS is currently working with the Police Department to develop a crime mapping system similar to CompStat. As a result, DTIS is capable of deploying relatively quickly CitiStat technology by utilizing and expanding existing infrastructure and experience.^v

Other Jurisdictions

In addition to Baltimore CitiStat, the cities investigated for their use of data management tools to improve performance include New York, Chicago, and Phoenix. Each city has implemented data management tools to varying degrees and has utilized this data differently than CitiStat.

The City of Baltimore

The Baltimore City Government launched in 2000 an extensive implementation of a statistics based performance management system for the review, analysis, and improvement of department performance called CitiStat. Again, CitiStat is not a technology program on its own but leverages technology for the purpose of improving department performance. Participating departments attend bi-weekly or monthly CitiStat meetings and present performance data to a panel composed of the Mayor or Deputy Mayor and city executives from Finance, Information Technology, and Public Works among other departments.

The Baltimore City government lists 11 "operational service agencies" that perform specific and standardized functions that participate in CitiStat. These departments include, among others, Public Works, General Services, Health, Housing and Community Development, and Parks and Recreation.

While these departments attend CitiStat meetings with the city's executives the departments also hold meetings internally to compile, review, and analyze performance data from divisions within the departments before attending CitiStat meetings.

The CitiStat system has the following components:

Baltimore's initial costs for building the CitiStat meeting room, buying "off-the-shelf" mapping software, staffing and training, and setting up the system at the department levels are estimated at $420,000.^vi

However, other costs associated with the CitiStat model are more substantial. For example, the start-up and operating costs of CitiTrack (the "3-1-1"/CRM system) in fiscal year 2001 was $2.5 million. CitiTrack costs about $4 million per year, which includes a $92,000 per month payment to Motorola to operate and manage the call center.^vii

The City of Baltimore has realized over $70 million in savings since CitiStat's inception due to reduced overtime costs, elimination and reduction of redundancy of programs, generation of new revenues, and increased efficiency. For example, CitiStat meetings revealed a source of inefficiency in the Public Works department involving a disconnection of tasks performed by two separate divisions within the department. Integrating and streamlining the task-fixing and replacing a water meter-resulted in savings of $5 million per year.^viii

Other notable achievements include the following:

The City of New York

The New York Police Department (NYPD) implemented CompStat in 1994 to realize decreases in criminal activity and efficiencies in police work. The CompStat approach to crime reduction involves gathering accurate, real-time crime data and developing effective tactics to reduce crime. The NYPD rapidly responds to crime indicators and tracks fluctuations in indicators. The following year in 1995, the city's Department of Corrections adopted a CompStat-like approach to cover all aspects of departmental management and operations called Total Efficiency Accountability Management System or T.E.A.M.S.

The successes of these two performance management approaches-the NYPD's CompStat and the Department of Corrections' T.E.A.M.S.-prompted other departments to implement similar management systems collectively called the Citywide Accountability Program (CapStat) over the last ten years. Unlike CitiStat which relies on performance meetings with City officials, CapStat focuses on internal meetings within the department staff and management, although performance indicators are released to the public on the CapStat web-site.^ix

The New York City Human Resources Administration (HRA) is an example of how a CapStat system was converted to meet the specific goals of the HRA. The HRA successfully converted welfare agencies into job placement centers using data management tools called JobStat, "the incorporation of data collected and processed from various sources to create JobStat monthly reports."^x

JobStat reports enable HRA directors, regional directors, and staff to examine job center and employee performance by tracking indicators on the timeliness of public assistance, percentage of errors on food stamp applications, fair housing request rate, customer satisfaction rate, etc. Monthly meetings are held with job center directors to review performance on such indicators.

In addition, by city charter mandate, the New York City Mayor's Office of Operations publishes annually the "Preliminary Mayor's Management Report" and the "Mayor's Management Report" that assess department performance in April and September of each fiscal year. The current administration is considering integrating CapStat with the Mayor's Management Report. The management reports include, among other things, performance goals, performance measures, and the results of the report are discussed at public meetings following its publication.

The CapStat system has the following components:

Challenges to CapStat include the following deficiencies:

Information about the costs of launching and cost-savings derived from CapStat are unavailable.

In March 2003, New York implemented a centralized customer service "3-1-1" call center that handles citizen requests for service and information. The call center uses a Siebel Customer Relationship Management (CRM) system that is the technology tool to manage processes and data. New York is currently working on expanding the CRM to allow all city departments to connect or use the CRM tools. The overall budget for New York's CRM system was $25 million. This figure includes costs of technology, procurement, and staffing. The costs of integrating departments is expected to equal or exceed the initial $25 million investment. Data analysis, performance management, and resource allocation are not yet developed although New York has already discovered process efficiencies.^xi

The City of Chicago

Chicago's "3-1-1"/Customer Service Request System (CSR) has similar components to the systems in place in New York and Baltimore. Chicago's CSR system provides a single point of access for government customers that want to request information or services from any of the city departments. The CSR system is Chicago's single work order management system and is capable of generating reports on any of the services performed by each department as well as mailing "customer service letters" upon completion of the work order.

In 1998, Chicago implemented its "3-1-1" call center and Customer Service Request system to enable Chicago residents to request City services more easily and enable the City departments to respond to service and information requests more efficiently. The City of Chicago utilizes the statistics based reports generated by the CSR system to monitor crew and service performance by department. However, instead of holding regular accountability sessions as in the Baltimore CitiStat approach, Chicago's Director of City Services meets regularly with department representatives to analyze department reports generated by the CSR system.

The City of Chicago invested $8 million to set up its "3-1-1"/CSR system and to integrate departments into the system. Estimates of the cost-savings to Chicago since implementing CSR are unavailable but range in the millions, according to city officials.

For example, the CSR system has helped reduce redundancy in issuing work orders utilizing the "duplicate check method." By tying the city's electronic grid sub-system into one grid, only one crew from the Bureau of Electricity is dispatched when multiple requests are made for utility services originating from the same area. As a result, the city estimates savings of $6.9 million in avoiding duplicate dispatches.^xii

The City of Phoenix

Phoenix-a City Manager form of government-has adopted incrementally a "managing for results" system since 1990 as part of Phoenix's commitment to "meeting residents' need for quality government services." The Phoenix City Auditor Department works with departments, the City Manager, the City Council, and citizen focus groups to develop performance and/or results indicators that reflect department inputs, outputs, efficiency, and outcomes.

The City Manager's Office evaluates and approves each department's plan for performance measurement and the Auditor biannually surveys all departments to analyze and evaluate each department's focus on performance results and reporting. As an incentive for departments to meet performance standards, the Phoenix "Performance Achievement Program" compensates executive and middle managers on a "pay-for-performance" basis.

The Auditor, city officials, and all 25 departments utilize automated database and information technology systems to collect and report performance results information. Each department electronically reports statistics monthly to the City Clerk and the City Auditor, and the statistics are turned into graphic illustrations contained in the City Manager Executive Report available online at www.phoenix.gov. Performance information is also available in the following formats: a one page weekly "City Page" in the Arizona Republic newspaper, weekly Council Sessions and Subcommittee Meetings, monthly "Notes Newsletter" distributed as an insert in Phoenix water bills, the "Annual Citizens Report", and the "Community Attitude Survey" that is conducted every two-three years.

An example of a department that has benefited from the "managing for results" system is the Phoenix Human Services Department (HSD). This department has fully automated its data collection process, improving department performance. Instead of relying on data entry staff, HSD inputs data about clients, services, and programs at the line level with the case worker sitting across from the client. Automation has enabled the HSD management and city officials to monitor programs on a month to month basis with the level of services and clients remaining fairly constant. In addition, HSD no longer depends on data entry staff to input data.^xiii

Analysis

Based on the experiences of the cities surveyed, San Francisco may want to consider adopting either of the two options listed below or implement a system that combines elements from both options.

1. The most feasible and useful approach for San Francisco is to upgrade and integrate departments' database systems to make performance data accessible to City officials, users, and the public. The City could mandate a format for reviewing the performance of each department according to selected performance indicators, like the CitiStat and CapStat approaches. San Francisco could formulate enforceable objectives for all departments and hold regularly scheduled meetings with department heads and representatives to discuss progress in achieving selected objectives for each department.

2. Although not required for performance management systems, San Francisco could implement the above option and additionally establish a centralized call center (e.g. Baltimore, New York, and Chicago's "3-1-1" telecommunications system). The call center would handle all customer service and information requests. The call center could feed data about the service delivery performance of departments (e.g. fixing potholes, picking up trash, etc.) into the City's performance management system.

Arguments:

Con:

CONCLUSION

The survey of cities shows that the jurisdictions examined implemented a performance management system and leveraged technology (e.g. database management systems, Customer Relationship Management systems like "3-1-1", GIS mapping, etc.) to enhance department accountability and improve the performance of participating departments. The jurisdictions also reported cost-savings, although such savings are not guaranteed outcomes.

San Francisco already has a means to measure the performance of departments and has some accountability measures, such as holding regularly scheduled public meetings and the annual budgeting process. A statistics based performance management system similar to CitiStat, however, will shift the culture and environment of the departments to a more performance and outcomes orientation.

CitiStat is a fundamentally different approach to performance management as it requires department heads and representatives to track, explain, and develop strategies to tackle real-time fluctuations in selected performance indicators. Technology tools allow the process to occur continuously based on customer need and immediate performance rather than once a year for budget purposes. Finally, an approach similar to CitiStat requires effective leadership to ensure that departments are held accountable for performance.

RECOMMENDATIONS

Due to current fiscal constraints it might be best that San Francisco implement a uniform and systematic method of reviewing the performance of city departments through bi-weekly or monthly performance meetings within departments and meetings with city officials. While this option will require a significant up-front investment in time, resources, and money, the cost-savings from introducing a more timely accountability mechanism into the management of departments may be significantly larger than the initial investment.

The costs of integrating several department databases into a common database, utilizing servers to transmit and consolidate data, and developing an application and web browser that makes data easily accessible and usable are not yet available as participating departments will have different needs. However, Department of Telecommunications and Information Services (DTIS) estimates that the time frame for developing a CitiStat mapping application based on existing infrastructure is approximately six months, not including process and management restructuring or complicated systems integration.^xiv

In addition, the Legislative Analyst recommends that the Board of Supervisors may wish to consider hiring an internal or external organization to study the feasibility, cost, and time frame of adopting a uniform data management system and integrating the system with a centralized call center during better economic times.

Appendix

Table 1. Performance management systems and accountability mechanisms in San Francisco and the four jurisdictions examined.

Appendix

Table 2. Page 1 of 16 from Baltimore's Department of Public Works - Water and Wastewater, May24, 2003 through June 6, 2003.

References

ⁱ J. Kost, "Performance-Driven E-Government and CRM," Gartner Research, September 18, 2003.

ⁱⁱ Ibid.

ⁱⁱⁱ Anne Jenkins, Office of the Controller, email, 10/10/2003.

^iv Ibid.

^v H. Seick, DTIS Telecommunications & Policy Advisor, email, 10/16/2003.

^vi H. Seick, DTIS Telecommunications & Policy Advisor, meeting, 9/26/2003.

^vii Elliot Schlanger, Baltimore Chief Information Officer, phone interview 9/25/2003.

^viii Ibid.

^ix CapStat information is available at the following website: http://www.nyc.gov/portal/index.jsp?catID=1724&pageID=nyc_stat_reports&9...

^x "Welfare Peer Technical Assistance Network: New York City Site Visit, JobStat: Using Data to Achieve Outcomes," Dec. 12-13, 2002, available at: www.calib.com/peerta

^xi H.Seick, email, 10/16/2003.

^xii Theodore O'Keefe, Director of City Services, phone interview, 9/30/2003.

^xiii Steve MacFarlane, Director of Planning and Policy, City of Phoenix Human Services Department, phone interview, 9/22/2003.

^xiv H. Seick, meeting, 9/26/2003.

^xv Cost information is not available for New York, Chicago, and Phoenix because those cities implemented performance measurement tools incrementally from 10 to 3 years.

^xvi Annette Hines, Deputy Commissioner 3-1-1, phone interview, 9/22/2003.