The Audit recommendation on EMS Service Configuration is to cut service levels by eliminating 13 FTE. This recommendation will reduce coverage by ambulances, far and away the busiest units in the Department, by at least 14% to 24% for the 16 hour period from 9PM to 11AM. The trade off is a minor increase of 9.5% in coverage between the hours of 2 PM and 6 PM. The Audit does not identify which neighborhoods are targeted for service delivery cuts.

The Department strongly rejects this proposal. Elimination of ambulances will increase response times during the time of day when patients are sickest. Advanced Life Support response times will further increase if the audit recommendation to convert three ALS ambulances to BLS ambulances is implemented. Quite simply, less paramedic staffed ambulances equals longer ALS response times. Equally important, workload for paramedics will increase during the late night hours when increased workload has the greatest impact on fatigue.

The Department is very concerned about the potentially misleading statement on page 4.2-24 that "Our recommendation would result in an annual cost savings of approximately $1,284,699 compared to the Department"s proposed One and One Response Program, and would distribute workload among ambulances more evenly. Our recommendation would also free-up 13 additional FTEs." It is not possible to realize this cost savings without eliminating the budgeted positions of field personnel i.e. one or the other - save money or free-up FTE - not both.

This section of the Audit states that there has been no significant improvement in EMS system performance, as measured solely by response times over a 21-month period of time. The Department believes this conclusion is incomplete and based on faulty statistical analysis. The report also contains inaccuracies about the numbers of FTEs assigned to EMS, ambulance workload and distribution, and efficiency of "short shift ambulances". This introductory section provides background information that will present a clear and accurate picture of the Department"s EMS performance over the past four years.

The primary goal of the 1996 EMS reconfiguration process was to reduce Advanced Life Support (ALS) response times by increasing the number of ALS units in the City through deployment of paramedic engine companies. Fire Department ALS engine companies are now a critical component of most large EMS systems in California as well as in many other areas of the country. Additionally, in the last several years, ambulances staffed with one paramedic and one EMT have become the standard of care in most EMS systems in the state. The audit ignores a great deal of testimony and documentation that was prepared and submitted on the need to implement this type of system in San Francisco. Instead the audit recommends re-implementation of basic life support (BLS) ambulances despite significant evidence that this is both legally difficult and unwise. The audit also recommends changes in ambulance shift schedules that will be both inefficient and unpopular.

Finally, the Audit does not analyze the current EMS system in relation to the pre-merger EMS system. As discussed earlier in this response, the Department has made significant improvements in every area that was identified as a problem prior to the merger.

In the introduction of section 4.2 - EMS Service Configuration, the Audit states,

The Fire Department has incurred $2,590,000 in annual salary and fringe benefit costs for 36 new EMS positions to staff four additional ambulances since FY 1999-2000. During this period, although workload per ambulance has declined, workload continues to be unevenly distributed among ambulances and ambulance response times have not shown consistent improvement¹.

In effect, the audit appears to be asking, "What is the City getting for its money?" This report will demonstrate that:

· Ambulance response times for all EMS calls have decreased throughout the City.

· Ambulance resource availability has improved

· Ambulance workload has been reduced

· Ambulance workload is appropriately distributed

Response Time Analysis

The Audit relies primarily on response time statistics to support its contention that there has not been significant improvement in the EMS system. Many other performance measures such as reductions in EMS claims and complaints, improvements in training and quality improvement, and increase in ALS resources provide important evidence about system performance.

Accurate and appropriate analysis of the SFFD EMS system based on Computer Aided Dispatch system (CAD) data is difficult for a number of reasons:

· The EMS system has been in flux for a majority of its existence. Personnel issues, changes in response configurations, and changes in unit deployment all contribute to complications of analysis.

· Changes in dispatching. In April 2000, the Department"s dispatch functions were reassigned to the Emergency Communications Department. This included a change in CAD systems. This change significantly affects the CAD data analysis process, and is referenced repeatedly throughout this report. The effects of this change are discussed later.

· The inherent limits of CAD data. When analyzing data, one can see what happened, but not necessarily why. One must often look to factors in addition to response times in order to understand what the CAD data represents.

The ability to draw a true and accurate analysis of the information presented by CAD data requires more than mere "number crunching." One must understand EMS systems, operations, and principles. One must also understand the sources of the data, and how those sources affect the ability to draw conclusions. Without that background, it is difficult to provide an accurate analysis on which to base future decisions.

The Department"s response utilizes more accurate and comprehensive response data and applies consistent analytical methods to CAD data starting with the month of January1998through December 2001 to demonstrate that the performance of the Department"s EMS system, including response times, has improved significantly. Notably:

· Response times are down, despite a change in CAD systems in April 2000 that increased queue times by several minutes. Measured at the 90^th percentile, citywide code two response times for December 2001 decreased 18:37 (52%)² versus January 1998"s baseline.

· Citywide code three response times decreased 3:15 (23%) for the same period, and code 3 life-threatening incidents responses are down 2:38 (19%).

· Unit availability has increased dramatically. In January 1998, the Department had at least 644 occasions where there was no ambulance available to send to a medical incident. Private ambulances had to be dispatched 462 times by the Fire Department. In December 2001, there were no occasions where the Department had no ambulance available, and private ambulances were utilized only 231 times.

· Medic unit utilization has decreased. In January 1998, the average medic unit spent 9 hours 50 minutes and 24 seconds per 24 hours responding to incidents and providing medical care. The same figure for December 2001 was 7 hours 3 minutes and 26 seconds. This change gives the crews time to tend to other responsibilities, such as apparatus cleaning and maintenance, restocking, training, and self-care (meals, rest, etc).

Much of the data cited by the Audit came from the Fire Department. At the time, it was the best available data. However, even then, there were significant limitations to that data set. These limits were identified to the Auditors at the time. Limitations of this data set and CAD data in general have been discussed repeatedly with representatives of the Budget Analyst. As noted earlier, some data sources cannot be verified, especially those relating to pre-merger performance. A full discussion of this issue follows later in the report.

Methodology

In order to obtain a true picture of the Department"s EMS performance, consistent analytical methods have been applied to five samples of CAD data. The month of January 1998 was selected as a baseline. It is the earliest full month for which raw CAD data is available, and is the closest approximation available of the operational system used by the DPH Paramedic Division prior to the merger. Four additional months were included: January 1999, January 2000, January 2001, and December 2001. The month of January is used as much as possible in order to minimize seasonal variations in call volume. Months were selected instead of fiscal years in order to provide a clear picture of system performance, unclouded by changes in system parameters that have occurred in the course of the years. Data is consistent from month to month within the time frames during which the various systems were deployed. These months allow for accurate and statistically relevant measurement of the Department"s progress in meeting the demands of its EMS mission, and for evaluation of its use of additional resources.

Unit response data for each month was downloaded from the CAD system into a Microsoft Access database. Responses for all ambulance types, including private ambulances and "phantom" ambulances (see next section for description), were queried from the database, and sorted by ambulance type (SFFD, private, etc.). The queries were limited to incidents triaged using the Criteria Based Dispatch system, the medical incident screening system utilized by the Department throughout all months studied.

The Department"s responses were then sorted by call category. Because there are several instances every month of multiple ambulances responding to a single incident, responses were indexed by incident number and sorted by "on-scene" time. Response data for later arriving ambulances was excluded from analysis. Responses for which no "on scene" time was recorded in the CAD also were excluded.

In order to permit geographic analysis of Department response, the resulting data was sorted by first-due engine for the location in question, using Department street box data. Times for each area are reported at the 90^th percentile for total response time and its components: queue time and roll time. Times are further reported geographically by Department Division and Citywide.

In addition to assessing response times, resource availability was also evaluated. By comparing utilization of private ambulances and phantom ambulances for the months in question, conclusions can be drawn regarding resource availability.

When the Department does not have an ambulance available to send to an incident, it will send an ambulance from one of the City"s private Advanced Life Support (ALS) providers, if one is available. Each one of these responses is recorded in the CAD, and can be counted.

If the Department has no ambulance resource to send to an incident, the CAD will generate a "phantom" ambulance, and will recommend the phantom unit with other companies for dispatch to a medical incident. These phantom units serve as a reminder to the dispatcher to send the next available ambulance to the incident, while allowing for dispatch of other responders to care for a patient until an ambulance is available. Dispatch of phantom units is also recorded in the CAD.

Average daily unit workload for medic units was evaluated. Average daily unit utilization was compared for the months in question. Utilization is defined as the time interval between dispatch of a company to an incident and availability of that company for dispatch to another incident. The unit utilization figures provide a partial picture of ambulance activity. Areas not addressed by this figure include mandatory training activities, unit restocking and maintenance, and time required for the return of the unit from the hospital to its service area.

EMS System Descriptions

This section of the Department"s response examines Department EMS responses for five months: January 1998 -2001, and December 2001. January 1998 provides the earliest baseline available from SFFD CAD data. January 1999, 2000, and 2001 examine the effects of 24-hour ambulance staffing and the BLS tier; by using January, potential seasonal variations in call type and volume are minimized. The current system of twenty 24-hour ALS ambulances is examined using data from the most current available month, December 2001.

The following sections describe the system as configured for each of these months. Discussion includes CAD system in use, number and types of units deployed, and personnel availability.

January 1998 - EMS Call Volume Increases

January 1998 is the earliest period for which Department CAD data is available. A number of factors led to a marked increase in EMS call volume during this period that are not discussed in the Audit. The Department of Public Health represented that it responded to approximately -165 EMS calls per day. When EMS dispatching became the responsibility of the Fire Department, EMS calls almost immediately rose to an average of 200 or more calls per day. This was due to use of more standardized triage and dispatch criteria by the Department. Additionally, the Fire Department did not endorse the common pre-merger practice of triaging 911calls for medical assistance out of the EMS system or deferring calls for service. Changes in health care financing, homeless services, seasonal flu and other illnesses also contributed to this increase. This is evidenced by the well-documented increases in Emergency Department visits throughout other parts of the country.

As part of the merger of the DPH Paramedic Division into the Fire Department, DPH and Fire Department dispatch functions were consolidated at SFFD Central Fire Alarm System (CFAS) in mid-December 1997. Ambulance crews were still on the DPH schedule of 10- and 12-hour shifts on a 40-hour workweek. Up to sixteen units were deployed out of static fire stations at peak times, based on deployment models developed through data analysis earlier in the merger planning process. This represents a change from the previous dynamic deployment model used by the Paramedic Division that improved coverage for the outlying neighborhoods.

The Department deployed an average of 10 to 12 ambulances over the 24-hour period depending on crew schedules and personnel availability. Due to staffing shortages, unit closures were not uncommon. Since all crews were on short shifts, identifiers later used for short shift units (e.g. 81, 92) are only rarely seen in the CAD data dispatching and communications were the responsibility of the Fire Department during this period. The Department used the PRC CAD system to manage unit dispatches and responses. Use of the Criteria Based Dispatch (CBD) system to triage and code medical incidents had been implemented in December 1997. The PRC CAD calculated queue time starting when the call-taker sent the incident to the system controllers for unit assignment. Queue times reported from this system report only the dispatch segment. The call-taking segment, including the sometimes-lengthy CBD triage process, is not included in these calculations. This greatly, and artificially, shortens overall response time calculations when compared to those from the CAD system used in later months.

The increase in call volume exacerbated response problems related to resource shortages inherited from DPH. As noted earlier, if the CAD does not have an ambulance available to recommend, it will recommend a phantom, or "X," unit. Dispatchers will send an ambulance from one of the private ALS providers, if one is available. Both of these types of dispatches are recorded in the CAD, allowing for evaluation of the Department"s resource allocation.

January 1999 and 2000 - 24-Hour Shifts and BLS Ambulance Deployment.

Paramedics were deployed onto 24 hour shifts on February 21, 1998. Due to the higher than anticipated workload on ambulances, the Department began to deploy BLS ambulances in August of that year. The data for January 1999 and 2000 reflects the activity of 20 ambulances staffed on a 24-hour basis. These months, the Department regularly deployed sixteen 24-hour ALS and four 24-hour BLS units. Occasionally, ALS units were converted to BLS due to paramedic staffing shortages. In addition, one to two short shift civilian ALS units were deployed during the week, depending on staff availability. Though uncommon, there were some unit closures if staffing shortages could not be addressed.

January 2001 - BLS Tier Phase-Out and a New CAD

This was the last month of BLS tier deployment. It was phased out at the end of the month, with ALS units taking the place of BLS ambulances.

In April 2000, the Department"s dispatch and communications functions were transferred to the new Combined Emergency Communications Center (CECC) at 1011 Turk Street. Along with the new facility came implementation of a new CAD system from Tiburon Inc. This CAD utilizes different time markers than the previous system. The Tiburon system begins logging dispatch queue time for an incident when the Automatic Location Indicator confirms that call location, generally as soon as the call-taker picks up the phone. By doing so, the queue time figures from this system incorporate both the call-taking and dispatching components of this figure, resulting in much longer queue times than those from the previous PRC system, which measured times starting when the call was sent from the call taker to the dispatcher.

December 2001 - ALS Ambulances and Limited ALS Engine Deployment

December 2001 is the most recent month for which CAD data is available, and reflects the current status of the system. The Department currently deploy nineteen 24-hour ALS ambulances on a daily basis. Utilizing the same number of paramedic FTE described in the Audit, the Department schedules four short shift ALS ambulances each day staffed by a combination of H1 paramedics and 2532 civilian paramedics. The Department staffed an average of five ALS engines per day, with a range of one to six ALS engines per day. No ambulances were closed or converted to BLS during this month.

Dispatch criteria and functions continue unchanged from January 2001.

Performance Measures

To address the Audit, the Department has analyzed a number of performance measures. They include:

· Response times and components

· "Phantom" unit dispatches

· Private ambulance dispatches

· Medic unit utilization time

Response Times

The Department has a responsibility to answer requests for assistance in a timely manner. Therefore, response times are important. The Department must meet response time standards mandated by the DPH Emergency Medical Services Section.

Response time analysis involves three measures: total response time, queue time, and roll time. Total response time measures the interval between the receipt of the initial request for assistance, and the arrival of units on scene. Queue time is the amount of time required to take the call and dispatch units. This interval is measured differently in the two CAD systems providing data in the study. The differences will be examined in more detail later. Roll time is the amount of time that units physically move from their initial locations to the location of the emergency.

This report focuses on roll times and queue times, but not total response time. The changes in CAD systems and the resulting changes in queue times, renders total response time invalid for purposes of comparison. Roll time is measured consistently between the two CAD systems, therefore comparisons will be meaningful. In addition, roll time is sensitive to the number of available units: more units mean each unit will have to travel a shorter distance to reach a given incident. Thus, roll time will reflect system activity and improvements throughout the period studied. Queue times for the first three months of the study are compared. These numbers all come from the same system, and thus can be analyzed. The queue time numbers in these cases is sensitive to unit availability as well if there are no units available, then calls must be held in queue until units become available to respond to those incidents.

Phantom Unit and Private Ambulance Dispatches

Another measure to consider is availability of resources. If the Department does not have ambulances available when the public calls for assistance, it will not be able to meet its obligation to those whom it serves. Response times are a partial measure of resource availability. The CAD provides two additional measures of ambulance availability: phantom unit dispatches and private ambulance dispatches.

When the Department does not have an ambulance available to send to an incident, it will send an ambulance from one of the City"s private Advanced Life Support (ALS) ambulance providers, King-American and American Medical Response, if one is available. Each one of these responses is recorded in the CAD, and can be counted.

If the Department has no ambulance resource to send to an incident, both CAD systems will generate a "phantom" ambulance, and will recommend it with other companies for dispatch to a medical incident. These phantom units serve as a reminder to the dispatcher to send the next available ambulance to the incident, while allowing for dispatch of other responders to care for the patient until an ambulance is available. Dispatch of phantom units is also recorded in the CAD.

Utilization Time

The final measure evaluated in this report is unit utilization time. The primary responsibility of the ambulance crew is to provide emergency medical care to those in need. Utilization time as tracked by the CAD measures the amount of time that the average unit is actually responding to requests for assistance. This is defined as the interval between dispatch of a unit to an incident, and the availability of that unit for another dispatch (generally when that unit clears the hospital after transporting a patient). One time segment that this measure does not include is the time spent returning to the station from the hospital. Depending on the area of the City in which a unit is stationed, this can be significant.

Responding to incidents is not the only responsibility of ambulance crews. They must also restock, clean, and maintain their vehicle in a response-ready condition. Firefighter Paramedics must participate in in-service training programs in order to maintain their skills as paramedics and firefighters. At the time of this writing, most paramedics are still in the field-training phase of their firefighter orientation. This confers additional training responsibilities for them and their company officers, requiring additional time during the day for in station training (drills). Finally, the crew must have the opportunity to eat and rest in the course of their watch. These are important workload measures that are not included in the unit utilization performance measures used in the Audit.

Response Times

The Department of Public Health Emergency Medical Services Section (SFDPH EMSS) mandates the following response time standards for ambulances responding to 9-1-1 calls.³ Times are defined at the 90^th percentile. The Department must meet or exceed the standard in each area. Exhibit 4.1.1 illustrates these response time standards:

Exhibit 4.1.1 SFDPH EMSS Response Time Standards

As noted here, there are three types of incidents for which the local EMS Agency sets standards:

·Code 2 incidents are the lowest priority of the medical incidents triaged by Criteria Based Dispatch. These are stable patients, for whom time is not a critical factor when ambulances are responding. The Department dispatches an ambulance without other first responders. Code 2 incidents comprise roughly 37% of all CBD-coded responses on a monthly basis.

·Code 3 incidents comprise the bulk of all medical incidents, consistently making up 60% of all CBD-coded incidents. These are incidents where there is a potential for danger to life or limb, and response times are important. These incidents require a paramedic response; BLS ambulances cannot be sent. These incidents also receive a first responder (typically an engine) in conjunction with an ambulance. Engines are sent to these calls because they can provide potentially life saving treatments such as defibrillation, CPR, and bleeding control.

·Code 3 Life-Threatening (LT) incidents are a subset of all code 3 responses, deemed by the SFDPH EMSS to be more emergent, and thereby requiring a faster response. These incidents include cardiac arrests, obstructed airway, and acute allergic reactions. Life-threatening dispatches make up a very small portion of the overall ambulance workload - less than three percent per month.

At this time, there are no response time standards for first responders (engines, etc.)

Queue, or dispatch, times are included in this analysis, even though the determination of the time interval changed in April 2000 (see January 2001 for details). These standards were written well before the implementation of the new CAD system, and have not been modified to account for the resulting changes. The following sections compare the Departments performance in meeting standards for each call type.

Over all, the Department has demonstrated remarkable improvement in meeting response time standards.

Exhibit 4.2.2: Ambulance Roll Times to CBD-Coded Incidents
90th Percentile - City Wide

Roll Times

Roll times provide the best picture of the Department"s utilization of its additional resources. Exhibit 4.2.2 below shows changes in roll times for all call types:

Exhibit 4.2.3: Ambulance Queue Times for CBD Coded Incidents
90th Percentile

The chart clearly demonstrates consistently decreasing roll times for each month examined. The only exception is for code 3 and code 3 life threatening incidents for January 2001. The reasons for the longer times that month are not clear from the CAD data. The increase in roll times for code 3 incidents that month (14 seconds versus January 2000) is not significant. As noted earlier, the number of code 3 life threatening incidents each month is extremely small. Therefore, several longer response times will affect the overall number.

When today"s system is measured against the baseline of January 1998, the improvements are remarkable. Roll times for code 2 incidents have decreased 24% (4:03). Code 3 responses decreased by 25% (2:50), and code 3 life threatening by18% (2:01).

Queue Times

The use of queue times for comparison is more problematic. As noted earlier, the two CAD systems that provide the data for this analysis measure this interval differently. Exhibit 4.2.3: below summarizes queue times for the months in question.

There are several notable features to this chart. First and foremost is the dramatic drop in queue times for Code 2 incidents between January 1998 and January 1999. This change is a direct reflection of the addition of the BLS tier. As noted earlier, queue times are a measure of resource availability, and this case serves as a strong example. With more ambulances to respond to all incidents, code 2 incidents are less likely to be held in queue while ambulances available to the system are responding to more urgent incidents. With BLS specific ambulances available to respond to code 2 calls, ALS units are freed up to respond to more acute calls in a more timely fashion, hence the decrease in queue times for code 3 and code 3 life-threatening incidents for the same periods. This analysis is supported by other indicators (see Resource Availability).

With minimal changes in the system between January 1999 and January 2000, there is less change in queue times. But there is a significant increase in this time interval between January 2000 and January 2001. This is due to the change in CAD systems, as discussed earlier. The minimal changes between January and December 2001 supports this analysis, especially for code 3 and code 3 life-threatening incidents, which are given higher priority for dispatch than code 2 incidents.

Resource Availability

The San Francisco Fire Department has an obligation to respond when the public calls for services that fall within its purview. In order to meet this requirement, the Department must have the resources available to meet demand. In addition to response times, CAD records contain two more data types that can be correlated to resource availability: phantom unit dispatches and private ambulance utilization.

"X" Unit Dispatches

To understand how "X" unit, or phantom ambulance, dispatches reflects resource availability requires some explanation about how these units are utilized by both the PRC and Tiburon CAD systems. When an incident is called up by the system controller for dispatch, the CAD system will recommend units based on the number and type of units required for a specific call type, status of those units in the system, and the closest available units based on geographic information in the CAD. For example, a code 3 medical incident requires the dispatch of an engine and an ALS ambulance, so the CAD will recommend the closest engine and ALS ambulance to respond to the incident.

If there are no ambulances to dispatch to such an incident, the CAD will recommend the closest available engine, and a phantom ambulance. The ambulance will show up on the dispatch line as "XXM1" or similar, hence the term "X unit." The dispatcher will dispatch the incident, sending the first responder engine to initiate patient care until an ambulance comes available for dispatch. The X unit serves as a reminder to the dispatcher that an ambulance must still be dispatched to the incident.

Phantom ambulance utilization will reflect resource availability for ALS level (code 3 and code 3 life-threatening) incidents. These are high priority incidents that require a first responder as well as an ambulance. X units generally are not assigned to code 2 incidents. These are lower priority calls that do not require first responders. They generally will be held in queue until all higher priority incidents have been dispatched, and there are ambulances available to dispatch to these incidents. Depending on the nature of the code 2 incident and expected holding times, an engine may be sent out to the incident to begin care and provide an update. In these cases, an X unit will also be dispatched, as with ALS incidents.

Exhibit 4.2.4 summarizes phantom ambulance dispatches per month for the months used for analysis:

Phantom Ambulance Dispatches per

Exhibit 4.2.4: Phantom Ambulance Dispatches per Month

The dramatic 99% drop in phantom unit utilization between January 1998 and January 2001(from 644 down to 5 per month) has two causes. First, the Department initiated crew changes in the stations on January 20, 1998. Prior to that date, ambulances returned to the Plant building for crew change, requiring greater out-of-service time and a significant loss of ambulance availability to the system.

After this change, the system still averaged 9.4 phantom unit dispatches per day for the rest of January 1998. The addition of four more ambulances to the system has caused the use of phantom ambulances to disappear. The availability of additional ambulances means that the Department almost always has an ambulance to send when the public calls.

Private Ambulance Dispatches

The other resource available to the Department during times of peak ambulance demand is private ambulance companies. American Medical Response (AMR) and King-American Ambulance have agreements with the county to make their ALS units available to the system when those units are not otherwise occupied. The Department regularly dispatches those ambulances to a number of incidents, but relies on them more in times of resource shortfall. Thus, the number of private ambulance dispatches is another measure of Department resource availability. Exhibit 4.2.5 illustrates private ambulance utilization over the last two years. If the Department has insufficient resources, the number of private ambulance dispatches will increase.

Private Ambulance Dispatches

Exhibit 4.2.5: Department Use Of Private Ambulance Dispatches

Overall, utilization of private ambulances has decreased. As noted earlier, there are limitations to the information that can be derived from CAD data, and the spike in private ambulance dispatches in January 2001 is one such case. Examination of CAD data, including CBD-coded incident volume and overall incident volume, does not suggest a reason for this anomaly.

Workload

Ambulance workload is the final indicator to examine. With more resources in the system, the utilization hours per ambulance should decrease. As noted earlier, ambulance crews have duties and requirements beyond responding to incidents. Utilization measures only one, albeit the most significant, component of ambulance workload.

Medic units (as opposed to BLS or civilian ALS ambulances) were selected for this portion of the analysis for several reasons. They are the unit type most consistently available for all three months in question. Staffed on a 24-hour basis, and capable of responding to any medical emergency, they are the workhorses of the ambulance fleet. Impacts of additional resources will be seen in the amount of time these units spend running calls. Exhibit 4.2.6 summarizes average medic unit activity for the months used for analysis:

Exhibit 4.2.6: Medic Unit Activity

These downward trends in Medic unit activity support other factors in showing the impact of additional resources on the performance of the Department"s EMS response system. With the additional resources, crews have more time to tend to their other duties, allowing them to maintain their units and themselves, resulting in better patient care. This has also led to a marked improvement in employee morale.

It should be noted that SFFD EMS call volume is still higher than the national average for EMS unit in major metropolitan Fire/EMS systems.

Analysis of Fiscal Year Response Times

Throughout much of the audit document, the Office of the Budget Analyst uses response time analysis to make their points regarding alleged inefficiencies in EMS system performance. As noted earlier, CAD data analysis for this system can be problematic for a number of reasons. In the Department"s analysis of the Auditor"s findings, it appears that there are multiple flaws in the use of CAD data for their analysis.

Audit Data Sources

Much of the data cited by the Budget Analyst came from the Fire Department. At the time, it was the best available data. However, even then, there were significant limitations to that dataset. These limits were identified to the Budget Analyst at the time. Limitations of this data set and CAD data in general have been discussed since with representatives of the Budget Analyst. As noted earlier, some data sources cannot be verified, especially those relating to pre-merger performance.

The Budget Analyst was given two documents containing CAD data from the Fire Department. The first was the Computer Aided Dispatch Data Report, Fiscal year 1999-2000, commonly referred to as the CAD data report. The introduction to that document noted several problems that must be considered when utilizing the data contained therein for system analysis. Chief among those were the changeover in CAD systems in April 2000, and the difference between calendar day and shift day.

The second document given to the audit staff was data entered onto a spreadsheet prepared by the auditors, including calculation functions. This spreadsheet contained summaries of EMS system performance data from January 1998 to March 2001. Auditors asked for, and were provided, figures on average and 90^th percentile total response times per month, as well as EMS-related incident and dispatch volume. When this spreadsheet was returned to the auditor"s office, the following caveats were included on the document:

· Figures for the period Jan-98 through Jun-99 are derived from previous reports. Methodology cannot be verified for comparison to data from Jul-99 forward.

· Figures for Jul-99 forward are derived directly from CAD data (except for columns "N" and "O").

· Daily deployment figures for Jan-98 through Jun-99 are not available. Therefore, "Average Daily Dispatches per Deployed Ambulance" figures for that period cannot be derived.

· Revised response time data for period Jul-00 through Mar-01 to correlate with data for FY1999-2000.

· Previous data from CECC CAD reports reflects "roll time" only; does not include "queue time" which is a component of the County-mandated response time standards.

· Similar methodology and reporting now for Jul-99 through Mar-01. Figures are in "minutes.seconds" (e.g. "10.12" equals ten minutes, twelve seconds).

The Department has made every effort to cooperate with the Office of the Budget Analyst, and assist in data procurement and analysis throughout the course of the audit. However, the department believes that the auditor has not presented the data accurately.

Measurement and Analysis is Problematic

As noted earlier in this report, measurement and analysis of the SFFD EMS system based on CAD data alone is problematic. The issues that must be considered and addressed include:

· The April 2000 cutover to a new CAD system that measures time intervals significantly differently from the previous. The net change has been extended queue times, which then extend total response times, giving the appearance of degradation of system performance where there is none.

· The EMS system has been in flux for a majority of its existence. Personnel issues, changes in response configurations, and changes in unit deployment all contribute to complications of analysis.

· The inherent limits of CAD data. Due to analysis limitations, it is currently impossible match up CAD data with the Departments shift schedule. Thus, a typical shift that begins and ends at 0800 hours will be spread across two days in a CAD system analysis. This becomes significant when one considers the issues addressed in the point above. In addition, when analyzing data, one can see what happened, but not necessarily why. One must often look to factors in addition to response times in order to understand the CAD data represents.

Another factor to consider is the sourcing of data. Data given to the auditor"s office was done so with some time constraints. As noted earlier, data from the earliest history of the system was taken from previously prepared reports. Differences in methodology can be surmised in that response times for code 3 and code 3 life-threatening incidents were combined in earlier reports, and broken out in those created by the Department specifically for the Auditor. This difference means that the data cannot be directly compared.

Current Analysis

In response to the audit"s allegations, the Department has prepared a new analysis of response time data, using the methodology described earlier in this report. All CAD data from January 1, 1998 to December 31, 2001 was included in this analysis. The reports are broken down by fiscal year, as the auditor uses. Keep in mind that the first and last segments are for six-month periods only. They are included for illustration of trends, and should not be taken as fully equivalent to the full fiscal year included. All response time intervals are taken at the 90^th percentile for the period cited.

Roll Times

Roll times are the most accurate response time component available to measure system performance over time. They are very sensitive to ambulance staffing and deployment, and are unaffected by the queue time issue from the CAD cutover. Exhibit 4.2.7 illustrates roll time trends for all three call types.

90th Percentile Ambulance Roll Times
CBD-Coded Incidents

Exhibit 4.2.7: Ambulance Roll times

In the three full fiscal years, there is a visible decrease in roll times in all call categories between each of the periods. There is a slight increase in code 2 roll times between the first half of 1998 and FY 1998-99. This may be due to an incomplete data set.

Queue Times

Exhibit 4.2.8 is similar to the one above, illustrating the progression in queue times.

90th Percentile Queue
CBD-Coded Incidents

Exhibit 4.2.8: 90^th Percentile Queue Times

The impact of the change in CAD systems is immediately apparent upon inspection, especially when examining code 3 and code 3 life-threatening incidents. These two incident types are better illustrations than code 2, because their higher priority means that they are dispatched immediately.

The changeover occurred in April 2000. There is marked downward trend in queue times for the first two periods, a slight increase in FY99-00, then a larger increase between FY99-00 and FY00-01. These changes show the impact of the CAD system, even over the two and one-half months at the end of FY99-00. Queue times remain relatively high for the last period examined, the latter half of 2001, confirming the impact of the CAD.

Total Response Times

Exhibit 4.2.9 shows total response times, taken at the 90^th percentile. These figures are inclusive of both roll time and queue time components.

90th Percentile Total Ambulance Response
CBD-Coded Incidents

Exhibit 4.2.9: 90^th Percentile Total Ambulance Response Times

Even to the uncritical eye, there is a marked improvement in code 2 total response times. It may also appear that there has been minimal or no improvement in code 3 and code 3 life threatening response. This is the reason that it is important to examine the response time components of queue and roll times. When one considers the impact of the CAD system changeover, it is apparent that the Department"s EMS system continues to improve its service to the City.

Comparison to Audit Findings

The Audit appears to make specific, identifiable errors when it claims that the Department"s performance has not improved, but has actually worsened.

On page 23 of the Introduction, the Auditor states, "Based on data provided by the Department, average ambulance response time increased between FY 1999-2000 and FY 2000-2001. Code 3 response times increased from an average of 10.56 minutes to an average of 11.57 minutes and Code 2 response times increased from an average of 20.29 minutes to 20.88 minutes." This assertion is repeated on page 4.2-12. Two issues appear with closer examination of the claim. In reviewing the data provided to the Auditor, it appears that the Audit is attempting to average the 90^th percentile figures for the months provided, rather than drawing a true 90^th percentile for all responses for the periods cited.

A more accurate examination of the figures, using the format of the auditor, shows the following:

· Code 3 total response times do increase, from 10.55 minutes to 11.38 minutes.

· Code 2 total response times increase from 19.40 minutes to 19.82 minutes. This figure is significantly shorter than those claimed by the auditor.

What the auditor apparently fails to account for, however, is the CAD system changeover. Roll times are a truer measure of system performance, and as the above chart shows, there have been continuous improvements in roll times. Therefore, contrary to the auditor"s conclusions, there have been improvements in system performance, given the extra resources.

On page 25 of the Introduction, the Audit repeats the response time claims made earlier, and notes in its footnote that there were indeed changes in response time measurement. However, that change was made during the periods cited by the audit. Nowhere does the Audit address this fact, and the impact of those changes on the claims it has mad.

On page 4.2-3 of the audit, the audit repeats the footnote from page 25, but cites the periods FY1998-1999 and FY1999-2000. If the audit is truly citing these periods, rather than FY1999-2000 and FY2000-2001, then the audit is attempting to compare figures from two different methodologies. As noted earlier, data provided to the auditors for the period prior to FY1999-2000 came from previously prepared reports. Those reports aggregated code 3 and code 3 life threatening response times, resulting in artificially shorter times for FY1998-1999 when compared to FY1999-2000. When one compares roll and total response times for those two periods in the charts above, across the board improvements in all areas are apparent.

Conclusions

The conclusions reached by the auditor regarding response times are inaccurate. Either through a lack of familiarity with the full intricacies of response time analysis, or selective analysis of the data provided, the Auditor has apparently reached erroneous conclusions regarding Department EMS performance. Even given the difficulties in examining larger periods of time noted earlier, it is apparent that the Department"s performance has continued to improve when looking at the performance figure that the Department has controlled during its stewardship of the EMS system: roll time. Because of the flaws in the auditor"s analysis, any recommendations based on its conclusions should be disregarded.

Distribution of Calls

Overall, the workload is fairly well distributed among ambulances and consistent with workload distribution in other Fire/EMS systems. The outer neighborhoods have fewer calls but relatively longer utilization times per call because the patients are sicker and the hospital transport times are longer.

The Audit states "CAD data for FY 1999-2000 shows that the number of medical dispatches is not evenly distributed among ambulances⁴." This is true in all EMS systems and is not a reflection of a poorly managed system. A majority of the City"s EMS calls are in the Downtown/Mission corridor. A higher percentage of the City"s ambulances are deployed in this area. But it is also necessary to deploy ambulances in outlying neighborhoods such as Hunters Point, Ocean View, Ingleside, Park Merced, Outer Sunset, and the Richmond where there is a lower call volume To better equalize distribution of calls, ambulances would have to be taken out of these neighborhoods and sent to the Downtown/Mission corridor. In the neighborhoods, the ambulance coverage is spread over a larger area and patients are sicker. Redeploying additional units downtown, where response times are currently the shortest, will greatly increase response times in the neighborhoods.

The Audit states :"In FY 1999-2000, the average number of ALS and BLS ambulance dispatches for 24-hour ambulances was six dispatches per ambulance per shift at Station 29 and approximately 17 dispatches per ambulance per shift at Station 1."⁵ The comparison is highly in-appropriate. The source data for this statement is the data provided in the 1999-2000 Computer Aided Dispatch Data Report, section D, Unit Performance Summaries. When one examines the information in that section, one discovers that the unit at Station 29 was in service for a total of 3 days during the entire fiscal year, one as a BLS unit, and two as ALS. The unit at Station 1, on the other hand, was in service 366 days (during a leap year) as an ALS unit, and an additional 31 days as a BLS unit. This is due to the difference between calendar and shift days, as noted in the CAD data report. As explained earlier, BLS units are available to respond to a limited number of calls. Therefore, they will have lower dispatch and utilization numbers when compared to ALS units, which can respond to all incidents.

The means the Audit appears to use to reach its numbers appears to be statistically flawed. Rather than attempt to draw a true average, in which one adds up the dispatches per day received by the unit at a given station, then dividing by the number of days of dispatches, the audit instead adds up averages to reach its conclusions

The Audit cites a 183% difference in the number of dispatches between multiple units at Station 29 and Medic 1 to justify its claim that workload is poorly distributed. This is an incorrect statistical analysis. The combined workload of two separate units, in this case an ALS and a BLS ambulance, that were not in service every day, cannot be compared to the workload of a medic unit that responds to a different type of call and is in service daily. A more appropriate analysis would be to compare the time on task of two equal units.

Using the same units chosen by the Audit: Medic 1, our busiest unit, and Medic 29, one of our least busy units, a comparison based on recent CAD data from the first 5 months of FY 01-02, shows the following:

Dispatch differential = 76%

Medic 1: average dispatches per day = 15.9

Medic 29: average dispatches per day = 9.02

Utilization differential = 39%

Medic 1: average time on task per day = 8 hours and 4 minutes

Medic 29: average time on task per day = 5 hours and 46 minutes

While there is a disparity between the call volume of downtown units and units in the outlying areas of the City, it is nowhere near as great as the Audit attempts to make it appear. Further, this is not a phenomenon unique to San Francisco. This disparity is found in every EMS system.

Short Shift Ambulance Performance

In section 4.2 of the report, the Audit states :

"Further, according to documentation provided by the Department, ambulances deployed for ten-hour shifts are dispatched to more medical calls, when averaged by the hour, than ambulances deployed for 24-hour shifts. Based on the FY 1999-2000 CAD data, the average number of dispatches for 24-hour ALS and BLS ambulances was 9.61 dispatches in 24-hours. According to a memorandum prepared by the Department, the average number of dispatches for ten-hour ambulances for the six month period from January, 2000, through June 2000, was 6.3 dispatches in ten hours."⁶

The auditor appears to be attempting to "selectively quote" from different sources, using different analysis methods, to prove a predetermined point. The 1999-2000 Computer Aided Dispatch Data Report examines average daily call volume for all units, including short shift ambulances.⁷ It finds the average dispatches per calendar day that short shift ambulances were deployed during this period to be 7.7 dispatches per day. The CAD Data Report states on page 2 of the introduction ("Work Day versus Clock Day"), "The standard calendar and clock day has been used," when referring to the difference between Department schedules and standard time keeping. With two 10-hour shifts per calendar day, the short shift units cited in the report averaged 3.85 dispatches per 10-hour shift. Accepting the figure provided by the auditor for all 24-hour units, when unit performance is compared on a dispatch per scheduled hour basis, 24-hour units are actually more efficient: 0.400 dispatches per scheduled hour for 24 hour units versus 0.385 dispatches per hour for shorter shift units. However, this comparison is inaccurate. By including BLS ambulances in the 24-hour unit mix, the auditor is artificially decreasing the call volume of the 24 hour ambulances when compared to an all-ALS fleet of short shift ambulances. A truer test is to compare Medic (ALS) units to short shift units. According to the CAD data report, Medic units were dispatched 10 times per 24-hours on average. Thus, the dispatches per scheduled hour comparison for ALS units would be 0.417 dispatches per scheduled hour for 24-hour units versus 0.385 for short shift units.

A similar conclusion is reached in the memo cited by the auditor in the second half of their attempted comparison. In brief, the Department memo compares performance of short shift units, which are deployed downtown, with the performance of the downtown units for the hours during which the short shift units are deployed. The Department memo finds that the 24-hour units are more efficient, when average dispatches per watch are adjusted to reflect disparities in work schedule and hourly pay between the higher-paid short shift paramedics and lesser paid 24-hour medics. When these adjustments are made, short shift units run 5.04 calls per shift, and 24-hour units run 5.5 calls for the same period.

Each of the documents cited by the Audit, when taken in whole, finds that 24 hour shifts are more efficient for the Department. The Auditor selectively quotes each document to attempt to prove an opposite, incorrect assertion.

In addition, 24-hour units are consistently available to the system. As a practice, short shift units are more frequently out of service for crew changes at the beginning and end of their watch. Department policy gives these units 20 minutes at the beginning and end of their watch for crew change. Thus, a short shift ambulance scheduled for 10 hours is actually available to the system for 9.33hours. Crews on 24-hour units do not have this luxury, and are expected to be available to the system on a 24-hour basis.

Scheduling and Coverage

The Audit fails to recognize the inherent difficulty of scheduling seven ambulances deployed on ten-hour shifts to cover the 16-hour period from 8:00 AM to 12:00 PM. At least three different shifts would be required. The charts that follow show that ambulance coverage is reduced by at least 14% to 24% for the hours of 9PM to 11AM. Coverage is the same for the 3-hour periods of 11AM to 2PM and 6PM to 9PM. Coverage is only increased by 9.5% for a 4-hour period from 2PM to 6PM.

Dynamic Deployment Schedule for 16 Hour Coverage with (7) 10-hour Units

Dynamic Deployment Ambulance Coverage Including the (16) 24-Hour Units

Current SFFD Ambulance Coverage

Compounding this problem is the need to change from the two non-overlapping shifts currently utilized to a three-shift overlap schedule. This greatly increases the difficulty of filling personnel openings created by absences. It is also more difficult to fill vacant shifts with overtime because most of the personnel are on a shift unavailable to work extra hours. A third shift also increases the out of service time needed for shift change. Finally, the effect of normal swings in employee absenteeism due to elective leave, sick leave, or disability is greatly magnified in a relatively small pool of people. Multiple shifts exaggerate this problem. The result is a greater likelihood of unit closures due to the short shift schedule.

The Auditor lists several reconfiguration options for staffing a combination of 24-hour and "short shift" units.

The Department has reviewed these recommendations and believes that the auditor"s proposal will result in increased service capabilities during four (4) daytime hours only, while reducing ambulance coverage during the remaining twenty hours of the response day.

Exhibit 4.2.10 compares the auditor"s staffing proposal to that of the Department and demonstrates the reduction in services as a percentage.

Exhibit 4.2.10: Staffing Model Comparison for Ambulance Response Capability

Conclusion of Introduction for Section 4.2:

The Audit examines EMS system configuration, and asks whether the Department is getting an adequate return on its investment in additional ambulances. The Audit"s conclusion appears to be "No." When one applies consistent analytical methods to CAD data from several months that illustrate the Department"s progression in developing its EMS system, one finds that indeed, the City is receiving a significant return on its investment. Response times have improved across the board, in most cases by 25% or more. The Department now has ambulances available to respond to requests for assistance when they are received; such was not the case in January 1998. The Department does not have to rely as greatly on the availability of private ambulance providers to meet the public"s needs. Ambulance crews are not being driven as hard as they were in the past, allowing them to meet their other obligations.

The Department has response time obligations mandated by county regulation. With the additional ambulances available to it, the Department now consistently meets requirements for code 2 incidents, and often meets code 3 requirements. Code 3 life-threatening incidents prove more problematic, but this will be addressed by the addition of a large number of ALS Engine companies. Almost all EMS systems with ALS response times that are under 8 minutes do so with ALS engines, not ambulances.

Clearly, there is still work to be done. Implementation of ALS first-responders will help the Department meet its obligations to the public through delivering a paramedic to the scene of medical emergencies sooner. On-scene incident triage by such responders will also result in decreased ambulance utilization, further increasing the availability of ambulances for incident response.

Recommendation of Auditor"s Report

4.2.1 Deploy 13 ALS ambulances and 3 BLS ambulances on a 24-hour basis and 12 ALS short-shift ambulances when implementing the One and One Response Program.

· Assessment of Department

The Department disagrees with the recommendation of the audit.

The Audit recommendation would cut service levels by eliminating 13 FTE. This recommendation will reduce coverage by ambulances, far and away the busiest units in the Department, by at least 14% to 24% for the 16 hour period from 9PM to 11AM.The trade off is a minor increase of 9.5% in coverage between the hours of 2 PM and 6 PM. The Audit does not identify which neighborhoods are targeted for service delivery cuts. ALS service would be further cut by converting three of the remaining ambulances to BLS units.

The Department has deployed ambulances on short shifts since the start of the merger in July 1997. In a 2001 survey of major metropolitan Fire-EMS systems, the Department found that one-third of the systems deployed short-shifts units to supplement their 24-hour shift scheduled ambulances, averaging 10 to 15% of deployed resources. Frequently these units are staffed through the use of overtime. The experience of this Department is that short shift ambulances are less efficient and more difficult to manage. Nonetheless, the Department recently moved all H1 paramedics to a 10 hour shift schedule, closed a 24-hour ambulance, and expanded the number of short shift ambulances it deploys to four per day. Currently the Department deploys 10% of its ambulance resources on a short shift schedule.

The audit uses inaccurate data when it claims that ambulances deployed on "short shifts" are more efficient than units deployed on 24-hour shifts (see earlier analysis in "Introduction"). It is critical to remember that personnel working short shifts automatically revert from the 48 to the 40 hour work week. This is an initial 20% loss of productive time. Directly comparing workload of the short shift units with 24 hour staffed units in the downtown area showed that the 24-hour shift crews were more productive than the short shift units. Reports were provided to the auditor on multiple occasions demonstrating that FF/PMs working 24 hour shifts on a 48 hour work week provide increased ambulance availability, which reduces response times to emergency medical calls.

Fire Departments throughout the United States schedule FF/PMs on 24-hour shifts. The Department is unaware of any fire department in the United States that routinely schedules firefighters on 24-hour shifts, and then schedules FF/PMs on an alternative "short shift" schedule. Clearly, other fire departments have found that 24 hour shifts work well from a scheduling and supervisory standpoint.

Operational Issues

The recommendation of the audit to reduce ambulance coverage by up to 24%⁸ would, if implemented, worsen working conditions for the crews that would remain on 24 hour shifts. The audit recommendations do not decrease the number of EMS calls. Fewer units would be available to respond to the same number of calls late at night. This will increase the workload of the personnel staffing these units. It is common knowledge that paramedics generally do not mind being busy during the day, but want to keep the post midnight responses to a minimum. This recommendation is sure to decrease paramedic morale.

This recommendation would also have the net effect of increasing response times. With fewer ambulances to cover the geographic area of the City during much of the day, and no change in call volume, it will take available units longer, on average, to respond to incidents. Patients at night tend to be sicker, requiring a more rapid response.

The Audit fails to account for the fact that the Department would have to utilize a new building to deploy a large number of short shift ambulances. Staggered start times for short shifts create scheduling problems that result in unit closures since paramedics on different units will frequently not have partners due to the use of sick pay, vacations, etc. To help address this, the short shift units must all be deployed from the same site to facilitate last minute changes in partners. The Department would be required to request additional funding for a new building. Managing this type of system requires additional personnel for scheduling and administrative rather than clinical supervision. The DPH Paramedic Division, which deployed units on short shifts, suffered from these same problems. This also led to unacceptably long ambulance response times as well as supervisorial problems.

Short shift ambulances initially deployed from a central location also create down time at shift change. Units do not deploy immediately at the start of shift due to restocking and checking of units that is an on-going activity with 24-hour staffed units. Short shift ambulances typically are unavailable for routine response during the last part of the shift so that units can return to the central location before the end of the shift.

The net effect of this proposed schedule change would be to decrease ambulance availability, increase ambulance response times, and increase costs of paramedic staffing, scheduling, and supervision; for no appreciable increase in available ambulance staffing.

The audit states that the drop in EMS call volume between midnight and 8:00 AM justifies the recommendation to educe ambulance coverage by up to 24%. The Audit fails to recognize that the reconfiguration proposal reduces ambulance at other times of the day as demonstrated by this report. The Department maintains that the present coverage is necessary to continue to provide the emergency medical care that the City of San Francisco has come to rely on.

Redeployment of BLS Ambulance Tier:

In addition to recommending that the Department deploy ALS units on short shifts, the audit recommends that the Fire Department deploy three BLS ambulances on 24-hour shifts. Reducing the number of ALS ambulances will increase ALS response times.

There are some benefits to the deployment of a BLS ambulance tier. The San Francisco Fire Department operated a BLS Ambulance Pilot Program for approximately 24 months from January 1999 to January 2001. EMS Division reports submitted to the DPH EMS Section during that period showed that the BLS Program was both safe and effective as a method of deploying additional ambulance resources in the City & County. In addition to providing improved ambulance service to the public through the addition of more units, the BLS Ambulance "tier" also provided operational, training and career development advantages for the Department"s firefighter/EMTs.

A comprehensive EMS Division report on the BLS Ambulance Pilot Program demonstrated that there were no significant problems in the clinical field performance of the BLS crews. EMS Section staff stated that they were concerned that Communications Center dispatchers were not accurately screening incoming 9-1-1 calls to determine if they should dispatch BLS versus ALS ambulances. However, a review of the data suggests that while dispatchers may not have always accurately screened the calls, they tended to err on the side of caution and consistently "over-triaged" calls, giving BLS-level calls to ALS ambulances. This resulted in a slightly lower utilization of BLS ambulances, but also helped ensure that BLS ambulances were not routinely being dispatched to calls for seriously ill or injured patients. In fact, the number of "ALS Upgrades"-calls in which BLS ambulances called for backup from an ALS unit-was very low, and compared favorably to cities such as Houston and Seattle which utilize BLS ambulance transport tiers. In conclusion, the Fire Department was satisfied with the success of the BLS Ambulance Pilot Program.

.

The BLS Pilot Program was closed primarily to address concerns from ambulance companies related to a possible violation of the Bronson Act. The Bronson Act states that any time the configuration of an EMS system is changed, the entire EMS system has to be put out to public bid. City representatives concluded that legal challenges and administrative issues, as well as possibly having to put the EMS service out to bid, would come at too high a cost. Fighting the Bronson Act would potentially cost the City millions of dollars in time and resources. City attorneys and DPH EMSS representatives directed the Department to discontinue the BLS Response Program.

Following that decision, the Department hired additional firefighter-paramedics as lateral hires from other fire departments. These FF/PMs were hired and deployed in January 2001, which gave the Fire Department the ability to convert the BLS ambulances to ALS units, resulting in no net loss of ambulance coverage for the City and improvement in ALS coverage.

Earlier analysis of the conversion of BLS ambulances to ALS demonstrates that the Department receives significant benefit from an all ALS fleet, especially when considering the Department"s response to the most seriously ill patients, for a marginal increase in cost. Under the Rapid Paramedic Response Plan, this represents the cost of one H3 versus the cost of one H2 receiving ambulance premium pay; a minimal net difference of seven percent.

While the Department is not opposed to the use of BLS ambulances as an additional resource in selective circumstances, the audit appears to minimize the substantial legal problems associated with redeploying the BLS Ambulance tier. Deployment of a BLS tier is decision that must be made by the DPH EMS Section.

· Department Action Plan and Timeline

The Department will not support any cuts to its ambulance fleet, the busiest units in the Department. The Department will continue to deploy a limited number of ambulances during peak times to improve service. The Department currently deploys all of its 2532 and H-1 Paramedics on short shifts. The Department has doubled the percentage of ambulances deployed on short shifts from approximately 5% to 10% of total EMS unit coverage. The Department will continue to monitor the performance and efficiency of short shift units. The Department believes that full implementation of the Rapid Paramedic Response System is the best option to improve Emergency Medical Services.

· Recommendation of Auditor"s report

4.2.2 Meet and confer with the respective employee organizations to implement 10-hour shifts for covered employees assigned to short-shift ambulances.

· Assessment of Department

The Department disagrees with the recommendation of the audit.

H3 Firefighter / Paramedics and H-2 FF/EMTs, are required to work 24-hour shifts under the City Charter and the IAFF Local 798 MOU. This MOU was just approved by the Board of Supervisors and will not be renegotiated for two more years. The City has no compelling grounds to reopen the work schedule provisions of the MOU. The 24-hour shift is an important work benefit for all FF/PMs and FF/EMTs. Forcing members of Local 798 to work a 4/10 work schedule would have a tremendous negative impact on morale.

The Audit fails to take in consideration that the entire work environment and culture of the Fire Department is built around 24-hour shifts. FF/PMs and FF/EMTs may resign en masse if forced to work these schedules that they will view as punitive. This could irreparably damage the Fire Department"s efforts to improve EMS service. Few if any members would volunteer to undertake paramedic training if it would mean they would lose the ability to work 24-hour shifts.

The Department is negotiating with Local 790 to memorialize the new short shift work schedules in their new MOU as a result of FLSA overtime issues.

Note:

The Department also views it as significant that the leadership of the Paramedic Chapter of Local 790, which has not been generally supportive of the 24-hour shift schedule, recently met with the Chief of Department to request that its members be allowed to remain on 24 hours shifts and not change to the 10 hour work day.

· Recommendation of Auditor"s report

4.2.3 Submit quarterly reports to the Fire Commission, evaluating the performance of the One and One Response Program, including ALS response times, number of ambulance and engine dispatches compared to number of medical events, workload distribution by ambulance, and attrition of paramedic staff.

· Assessment of Department

The Department concurs with the recommendation of the audit.

The Department is already required to submit monthly reports to the DPH EMS Section regarding the performance of the Rapid Paramedic Response System (RPRS) Pilot Program. These reports will also be forwarded to the Fire Commission for their review.

The Department believes it is worth commenting on the relevant portions of the audit that led to this recommendation.

Engine and Ambulance Co-response

The Audit states that an average of 1.58 units, including ambulances and engines, are sent to medical calls. The Audit uses this statistic to express a concern that "dispatching multiple vehicles to each medical incident increases demand on Department resources and could eventually lead to demands for increased staffing and associated costs." Multiple unit dispatch is standard practice in most EMS systems and consistent with DPH EMS Section regulations. The Emergency Communications Department dispatchers automatically send a "first-responder" BLS or ALS Engine along with an ambulance to all potentially life-threatening (i.e. Code 3 calls), which make up 60% of all San Francisco"s EMS calls. While many of these calls turn out after the fact to involve patients with minor injuries or illnesses, dispatchers cannot reliably ascertain this information over the phone. If the Department were to withhold the dispatching of first response engines to these Code 3 calls and only send an ambulance that might be much further away, the City would suffer a major increase in customer service complaints and greatly increase its medical-legal liability.

NFPA Standard 1710 mandates the response of two paramedics and two EMTs to all ALS level medical incidents. Short of placing four person crews on ambulances, this requires the dispatch of two companies to potentially life-threatening incidents.

The Department is convinced that the entire professional emergency medical community in San Francisco would oppose any system change that eliminates co-response to ALS level calls by the many engine companies strategically deployed throughout the City.

The auditor was provided a copy of the 2001 JEMS magazine survey of the 200 largest EMS systems in the nation, which shows that in over half of these systems first-responder engines are required to respond to all EMS calls, not just Code 3 incidents. These systems respond engines to all EMS calls as a safety measure to ensure that help is quickly available on scene in case the call was inappropriately under-triaged or the patient"s condition deteriorates.

The Department does recognize that in the initial phases of the Rapid Paramedic Response Program there will be an increase in co-response by first response units. This is because the sickest patients require the closest unit, ALS or BLS. In addition these calls require two paramedics. Due to the initially limited number of ALS first response units, some calls may require multiple first response dispatch. In these incidents, any additional units are quickly placed back in service after the patient is assessed to determine the level of care necessary. This problem will decrease as more paramedics are trained, hired, and deployed on ALS first response units.

The audit recommends that the "number of ambulance and engine dispatches compared to medical events and workload distribution by ambulance⁹" should be included in a report on the RPRS "One and One" Pilot. In fact, the Department closely monitors and routinely reports this phenomenon and does not view it as a problem.

Multiple Unit Dispatches

The audit indicates that there is a problem with duplicate ambulance dispatches to medical calls. It cites a slightly higher rate of dispatch increase of 0.07% compared to incidents from 1998 to 2000. This increase occurred during a time of great changes in the system including deployment of the BLS ambulance tier, addition of resources, and changes in shift schedules. The Department does not see this as an on-going problem.

Multiple ambulances are often dispatched to calls where there are multiple patients, or the potential for multiple patients, such as bus accidents. An ambulance that was dispatched to a low priority call may be "diverted" and sent to a more serious medical emergency. An ambulance may hear a dispatch for a unit to a nearby call and place itself back in service to respond from a hospital or when a patient declines transport in order to respond to the call. The original call is then sent a "duplicate" ambulance. These are routine occurrences in all EMS systems and represent an appropriate utilization of resources-ensuring that the most serious patients receive the fastest ambulance response.

Attrition Rate:

The audit states that the Department should monitor the attrition rate of paramedics. The audit reports the Department is experiencing "relatively high firefighter paramedic attrition rates¹⁰." In fact, the Department"s firefighter paramedic attrition rate over the last four years is under 5% per annum, which is lower than the national average of 6% for fire departments and far lower than the18% per annum for other EMS services.¹¹

It is also true that, due to changes in State and local pension plans, firefighter paramedics are frequently moving from one department to another to maximize job satisfaction, gain new promotional opportunities, take advantage of better home and educational values, or address lifestyle issues. Many fire departments are seeing high paramedic attrition rates.

· Department Action Plan or Timeline

The Department will submit monthly reports, beginning in February 2002, to the DPH EMS Section on the performance of the "One and One" Rapid Paramedic Response Pilot Program based on the criteria approved as part of the RPRS pilot program. These reports will also be forwarded to the Fire Commission for their review. The Department will forward these reports to the Board of Supervisors if so requested.

· Recommendation of Auditor"s report

4.2.4 Submit an annual report to the Board of Supervisors, as part of the annual budget review, evaluating the costs and benefits of the One and One Response Program, including sick leave and overtime use, ALS response times, and number of dispatches compared to number of medical events.

· Assessment of Department

The Department concurs with the recommendation of the audit.

The Department will submit a report on EMS issues based on specific direction from the Board of Supervisors. The Department is concerned that the data elements referred to by the Audit are overly broad and not discrete to the Department"s EMS services. The Department does not manage a separate EMS service that can be easily analyzed from the perspective requested by the Audit.

A number of questions must be answered in order to construct relevant reports. What is the EMS portion of the expense of an Engine company staffed with a firefighter paramedic or a firefighter EMT that responds to fires, rescues, and medical emergencies? How are disability and sick leave calculated for personnel who work on fire and EMS units? What is the allocation of training done by EMS staff who also provide mandated OSHA training, technical rescue training, etc.?

It will not be possible to submit a comprehensive report on the pilot RPRS pilot program as part of this budget approval cycle for FY 2002-2003. There will not be adequate data from a pilot system that has only been implemented for a few months. The Department will submit copies of preliminary reports if requested, and anticipates the collection of data and reports adequate for use in the budget approval cycle for FY 2003-2004.

· Department Action Plan and timeline

The Department will work with the Board of Supervisors to determine exactly what can and should be provided as part of an annual report on EMS services. The Department anticipates completion of this project by September 2002.

¹ Management Audit of the San Francisco Fire Department, 2001; pg. 4.2-2

² Time formats throughout this report are given in minutes: seconds, or hours: minutes: seconds. Thus, 18:24 should be read as 18 minutes, 24 seconds, and 9:50:24 as 9 hours, 50 minutes, and 24 seconds.

³ Ref: SFDPH EMSS policy 2120, section II. This policy includes response time standards and definition of code 3 life-threatening incidents

⁴ Management Audit of the San Francisco Fire Department, 2001; pg. 4.2-7

⁵ Ibid, Introduction pg. 22, repeated on page 4.2-8

⁶ Ibid, Introduction, pg. 22, repeated on pages 4.2-7 to 4.2-8. Auditor footnotes omitted.

⁷ 1999-2000 Computer Aided Dispatch Data Report, Unit Performance Summaries

⁸ Management Audit of the San Francisco Fire Department, 2001 pgs. 4.2-14 and 4.2-18

⁹ Management Audit of the San Francisco Fire Department, pg. 4.2-19

¹⁰ Ibid, pg 4.2-4

¹¹ JEMS magazine, October 2001, page 33