Planning and purchasing for information technology systems - processes which are linked through City department budget-development and priority-setting - are performed by each department in a manner specific to each department and independent of other departments, except in those cases when a specific information technology project is inter-departmental or use of a system is required for several or all City departments.

The City's 1996 Strategic Plan for Information Technology recommended that the City develop a current systems assessment which would include (a) an inventory of all citywide systems and (b) an inventory of all current projects and technical skills available citywide. However, eleven years later, in 2007, no such citywide assessment exists and, in fact, most City departments have neither (a) policies and procedures for inventorying existing information technology systems; nor, (b) a strategic plan that is either specific to their information technology systems or includes specific information technology objectives.

The only constant across departments is that each individual department has its own method to plan for information technology needs and does not work within a larger City framework unless technologically or legally required to do so. The Department of Telecommunications and Information Services provides professional and technical support to City departments only to the extent to which departments find these services useful. Responses received from departments for this report ranged from most departments stating that they rarely work with the Department of Telecommunications and Information Services unless they are required to do so, to at least one department stating a desire for greater assistance from the Department of Telecommunications and Information Services, if this particular department only had the funding available to pay for such assistance.

The City's Decentralized Approach to Information Technology Systems

Information technology systems have developed within departments on an individualized basis. This is largely the result of funding for information technology systems having been allocated on a department-by-department basis, which discourages collaboration among departments when systems or technology needs overlap. The consequences of such individualized systems development have been (a) inconsistent planning processes, (b) inconsistent project funding, which restricts departments' abilities to develop multi-year plans, and (c) cross-City redundancy in those information technology systems and processes which are not department-specific.

In addition to inconsistent development of information technology systems, the City lacks any centralized, coordinated communications mechanisms for (a) relaying information directly to information technology staff across City departments or (b) sharing information across departments regarding information technology-specific issues.

Inconsistent Information Technology Systems Strategic Planning Processes

Most City departments finance their information technology systems on a year-by-year, as-needed and as-funding-is-available basis. A planning process including an information technology-specific strategic plan would help departments identify overall departmental needs, prioritize those needs within a timeline framework, and establish baseline information technology expenditure needs on an annual basis.

Existing Strategic Plans

Approximately half of all City departments surveyed for this report state that they have a strategic plan, and a few more, including the Municipal Transportation Agency and the Planning Department, are in the process of developing their own strategic plans. These strategic plans, however, typically address only department-level goals, or client service-specific goals, while lacking specific, quantifiable information technology goals. The challenge for many departments, then, is taking department-level goals and using those to develop specific, quantifiable information technology goals, either through the inclusion of specific goals in a department-wide strategic plan or in through a separate information technology-specific strategic plan.

The Airport's FY 2007-2008 Efficiency Plan reviews service-specific functions by bureau and provides business strategies along with supporting five-year goals. Serving primarily as a business plan for the Airport, the Efficiency Plan does make specific mention of information technology in several areas, but it does not explicitly address the underlying information technology systems that are necessary in order to implement many of the recommended business strategies. Instead of using this Efficiency Plan for implementing appropriate information systems goals, the Airport sets and tracks information technology goals through a monthly checklist. These goals are set annually by Airport executive staff, in a process similar to that at Public Health as discussed below. Additionally, the Airport has an information technology-specific strategic plan which it is currently updating and expects to have completed by Spring 2008.

The Recreation and Park Department has a Strategic Plan for Information Technology that is approximately three years old. The Recreation and Park plan includes a detailed analysis of existing information technology systems and personnel, comparisons to other cities, a vision for future information technology investments, and an action plan, which includes specific task recommendations with timelines, estimated costs, and an identification of the departmental deficiencies which would be addressed by each recommendation. While this Strategic Plan for Information Technology is specific in its analysis and recommendations, Recreation and Park has not utilized the specific information contained in the plan to provide explicit direction for information technology planning and purchasing processes. Further, the Information Systems Manager for the Recreation and Park Department describes the planning process for information technology as 93informal,94 and the Strategic Plan for Information Technology has been used mostly as a document for background information, rather than for the Action Plan directives it contains.

It is not enough for a department to have an information technology-specific strategic plan. Such plan should provide specific, quantifiable goals within a timeline framework that a department can check against actual outcomes. In addition, the City should develop incentives and/or requirements that direct each department to re-visit its strategic plan as a means of ensuring achievement of strategic plan goals.

Resources for Development and Implementation of Strategic Plans

An additional, critical challenge to successful implementation of a strategic plan is having the resources to implement that plan. Departments have expressed difficulty identifying funding for strategic plan (re-)development, which indicates that departments would have even greater difficulty identifying funding to implement any strategic plan recommendations. Therefore, it is incumbent that the development of strategic plans for individual departments account for funding limitations, existing funding and staffing resources, and the ability of departments to obtain additional funding or staffing.

Inconsistent Information Technology Project Funding

Many departments reported an inability to predict total available information technology-specific funding from year to year. While most departments report some baseline information technology funding available for maintenance of existing systems, they also report an inconsistent funding availability for information technology improvements and upgrades from year to year, with a department-level preference generally given to service delivery over information technology improvements and upgrades. A few departments, however, report some funding consistency for their information technology improvements and upgrades.

The Department of Public Health, with 166 information technology personnel and a total information technology budget in FY 2006-2007 that exceeded $32,000,000, has a formalized information technology systems planning process. Public Health's planning process includes the following steps:

(1) Senior information technology managers interview division managers to identify needs, including implementation timelines and costs;

(2) Budget requests are prioritized by an Information Technology Steering Committee; and,

(3) A department-wide information technology-specific recommendation is developed by the Information Technology Steering Committee, including implementation timelines and costs, and submitted to a department-level budget review.

According to Public Health, funding is typically available only for those items determined to be "essential," which is the highest of four priority levels, from (1) essential to (2) high to (3) medium to (4) low; however, information technology-specific recommendations which go unfunded in the current budget year are carried forward for review and possible implementation in future years. As a result, items which get a lower prioritization in the current year, and thus go unfunded, may be given a higher prioritization in future budget years. This system enables Public Health to take a multi-year view of information technology improvements and upgrades and have some baseline budget funding expectations from year to year.

The Department of Public Health is an exception to the general rule. Most City departments' information technology divisions reported that they generally only pursue those information technology improvements and upgrades for which funds can be appropriated in any given year. This process of erratic and inconsistent funding from year to year has hampered departments' ability to develop (a) information technology systems strategic plans; (b) information technology project plan timelines; or, (c) quantifiable information technology objectives. While some departments have information technology systems goals, these goals do not necessarily correlate to specific, quantifiable objectives and projects with clearly-defined funding sources and amounts. As a result, departments' information technology goals are often under-funded and de-prioritized relative to departmental core service delivery objectives, unless such goals (a) can be quantifiably shown to improve core service delivery objectives or (b) are tied to a specific directive from the department's senior management and/or the Mayor's Office.

Two examples of planning being driven by funding availability are from Human Resources and the Assessor-Recorder. In FY 2005-2006, Human Resources, which currently has approximately 150 computers in use, purchased approximately 110 new computers. Human Resources leveraged the balance on its existing work-order with the Department of Telecommunications and Information Services to fund this replacement of 73.3 percent of the department's existing computers. Were it not for the available balance on the work-order with Telecommunications and Information Services, Human Resources claims that it would not have had the funding available for computer replacement. In FY 2006-2007, Human Resources has not replaced any computers, and funds for replacement were not included in the FY 2007-2008 budget.

The Office of the Assessor-Recorder, with approximately 120 computers, has replaced approximately 20 of these computers in the past two years, which is equivalent to a 12-year replacement cycle. The Assessor-Recorder received 31 new computers in the FY 2007-2008 budget.

Information Technology Purchasing Processes

As stated previously, City departments' planning processes are as varied as the funding sources, funding availability and core service delivery objectives of each department. In contrast to such widely disparate planning processes, citywide purchasing processes tend to be more consistent. This relatively better consistency for purchasing processes, as compared to planning processes, is a result of a centralized Office of Contract Administration which oversees all information technology purchases and professional services agreements, primarily through the City's Computer Store, which is managed by the Office of Contract Administration.

The City, through its Committee on Information Technology, established the Computer Store approximately 10 years ago, in order to fast-track the information technology purchasing process. The Committee on Information Technology managed the Computer Store until August of 2006, at which time management of the Computer Store was transferred from the Committee on Information Technology to the Office of Contract Administration.

All departments report using the Computer Store for the majority of their information technology purchasing requests. The Computer Store includes eight separate information technology vendor contracts that were approved in January of 2004 and are set to expire in December of 2007. Because the contracts have already been approved, departments do not need to proceed through the standard Request for Bids process, as they do for any purchasing requests that occur outside of the Computer Store. For departments, then, the advantage of using the Computer Store is that the contracts are already approved, which expedites the City's standard purchasing process.

The differences between these two processes are shown in the following table:

The primary difference in these two processes shown above is that the standard purchasing process is driven mostly by the Office of Contract Administration, while individual departments maintain greater control in the Computer Store process, as all questions about contractual obligations have been answered in advance of the purchasing request. As a result, the Computer Store purchasing process can take half the length of time as the standard purchasing process.

The Budget Analyst received copies of procurement manuals from several City departments and, based on a review of these manuals, notes these manuals are outdated and include information on information technology vendors that are no longer under contract with the Computer Store. The Budget Analyst recommends that the Office of Contract Administration review its procurement policies in order to ensure that departments have the appropriate information to make information technology procurement decisions and that processes are applied consistently across all departments.

Tracking Information Technology Purchasing Requests at the City Level

As part of the purchasing process, the Chief Technology Officer at the Department of Telecommunications and Information Services reviews all departmental purchasing requests made through the City's Computer Store in order (a) to ensure that such requests are needs-appropriate and compatible with existing City systems; and, (b) for those purchasing requests that involve professional services agreements, to address whether specific tasks are best performed by City personnel versus outside private contractors.

Given current systems, the City is unable to track departmental purchasing requests (a) over time or (b) against long-term departmental information technology goals. However, the Department of Telecommunications and Information Technology is developing a web-based, intranet system for automated tracking of Computer Store purchasing. Telecommunications and Information Services anticipates development of an additional component to this new system which will allow the tracking of specific metrics for the categories and types of acquisitions and the quantities of commodities.

What neither the existing system nor the pending automated process include are an ability to track purchasing requests against long-term strategic plans or specific department information technology goals. Therefore, it seems appropriate that this tracking system includes an additional component that would allow the Department of Telecommunications and Information Technology to track purchasing requests against individual long-term department information technology goals in order to ensure that purchasing requests are not only needs-appropriate but also goals-appropriate

Leasing vs. Purchasing Hardware

Most City departments purchase their information technology hardware outright and attempt to replace such hardware on a 3-year replacement cycle. The Department of Building Inspection, however, recently undertook what it has termed a "refresh" of all of its existing hardware by entering into a 3-year lease agreement for such hardware with manufacturer Hewlett-Packard. For approximately 335 desktop computers and 30 servers, Building Inspection is paying $325,000 annually. At the termination of the 3-year lease agreement, the department will have the option to either (a) replace all of the leased hardware with new equipment that the department can either lease or purchase, or (b) buy-out all of the hardware for $65,000. Given that the department intends to replace all printers and servers on a 5-year cycle, the department does intend to buy-out those specific pieces of equipment. For the desktop computers, however, the department intends to replace them on a 3-year cycle, which may be extended depending on financial constraints at the end of the current 3-year lease term.

While Building Inspection's "refresh" is not equivalent to an information technology-specific strategic plan, the 3-year term of the lease allows the department to have a baseline annual expenditure amount which accounts for the department's hardware replacement costs. Despite Building Inspection's recent investment in new technology, however, the department still finds itself unable to comply with a two years-old Civil Grand Jury request for the department's systems and documentation to be more open to the public, although the department states that it will be in compliance with the Civil Grand Jury request with the implementation of a new system that has an expected roll-out in June of 2008. In the absence of a strategic plan, therefore, Building Inspection has been able to implement a "refresh" project for hardware which accomplishes some of what would be expected from a strategic plan for the department's hardware, but the department has not had similar success for in its specific efforts in providing better public access to Building Inspection documentation.

Cross-City Redundancy of Information Technology Systems

While many departments utilize software technology from the same vendors, such as Microsoft, Novell, and Symantec, most departments individually pay for licenses with each of these vendors. For example, while nearly every City department utilizes Microsoft Windows or Symantec Anti-Virus software, nearly all departments have their own individual licenses for usage of this software. This redundancy in software licenses provides the City with an opportunity to negotiate lower license costs by aggregating its total information technology needs across departments. For example, rather than each department separately negotiating with Microsoft for Microsoft Windows licenses, the Committee on Information Technology could negotiate a single citywide license agreement which is then offered by the Computer Store to all City departments separately. Taking advantage of aggregated City information technology needs in this way is expected to result in net savings to departments because the City - as a single entity - could leverage its aggregate information technology needs better than individual departments, as some departments are only in need of a few licenses while others are in need of a few thousand.

The Oracle Enterprise License Agreement

From 1998 to 2003, the Department of Telecommunications and Information Services had a citywide enterprise license agreement with Oracle for several types of Oracle software. The Controller requested Telecommunications and Information Services to manage the process for a new enterprise license agreement with Oracle. Beginning in November 1997, Telecommunications and Information Services solicited feedback from City departments through phone calls and letters to departments' Information Technology managers. Despite this outreach effort, documentation provided by Telecommunications and Information Services to the Budget Analyst shows that, while Telecommunications and Information Services projected first-year need of no more than 728 licensed users, the agreement entered into with Oracle had a minimum requirement of 1,000 licensed users in the first year. Telecommunications and Information Services represents that the City had only 500 total users in the first year of the agreement. Additionally, documentation provided by Telecommunications and Information Services shows that the City was under pressure from Oracle to execute an agreement.

Therefore, while the stated basis for entering into a citywide enterprise license with Oracle was that Telecommunications and Information Services had determined that, over its five-year term, such agreement would result in a net savings in citywide licensing costs, the agreement resulted in a net increase in citywide licensing costs, as more licenses were purchased at the start of the agreement then were needed by City departments.

Telecommunications and Information Services maintains that departments either (a) insisted on negotiating their own licenses with Oracle or (b) bundled Oracle software into information technology-related requests for proposals, against the specific instruction in the Department of Telecommunications and Information Services memorandum that "Any application system being procured should not bundle Oracle software with the bid." Departments' responses have been that the Oracle enterprise license agreement (a) was entered into without adequate consideration of City departments' needs, (b) offered technology that was different than what departments needed in 1998.

Current Enterprise License Agreement Considerations

The citywide enterprise license model has recently been revisited at the monthly meetings of information technology personnel that take place in City Hall, and such model is also being reconsidered by the Department of Telecommunications and Information Services.

The Department of Telecommunications and Information Services is having talks with Oracle again about an enterprise license agreement with the software vendor. According to interviews with the information technology managers of two major City departments, the package of offerings being considered for inclusion in this pending agreement would be less-than-desirable. As a result, departments may prefer to negotiate directly with Oracle to address their needs, rather than utilize an enterprise license agreement, negotiated by Telecommunications and Information Services, that is less than ideal. Rather than directly engaging with departments to guide the agreement negotiations process, Telecommunications and Information Services is instead negotiating on its own with Oracle.

This method of negotiating with the vendor without directly engaging City departments creates an inherent disconnect between departments' needs and what Telecommunications and Information Services offers. In order to adequately address departments' needs with any new enterprise license agreement, Telecommunications and Information Services should have a process to continually solicit feedback from City departments while negotiating any enterprise license agreement, in order to determine and receive the most-appropriate technological offerings.

Sole Source Contracting

Departments reported utilizing sole source contracts when (a) specialized software was provided by only one vendor, (b) maintenance on particular software was best provided by the vendor of that particular software, or (c) a contract was renewed beyond the full term under the original contract, including extensions of such contract.

Each City department submits any proposed sole source contract to the Office of Contract Administration. Contract Administration has approval authority over each proposed sole source contract that (a) has a term of fewer than ten years, or (b) costs the City a total of less than $10 million; but any proposed sole source contract exceeding either or both of these threshold amounts is referred to the Board of Supervisors for approval. A key distinction between sole source contract review between these two bodies is that, while the Board of Supervisors considers policy appropriateness of proposed sole source contracts prior to approval, the Office of Contract Administration reviews proposed sole source contracts exclusively for their appropriateness as a sole source contract.

According to information provided by the Office of Contract Administration, the City expended more than $13 million on 90 separate information technology sole source contracts in FY 2006-2007, none of which surpassed the above-stated threshold which would have required Board of Supervisors approval. This estimate of $13 million represents 20 percent of the approximately $65 million on 405 total sole source contracts citywide in FY 2006-2007.

The Airport's Alcatel Contract

Since April 2005, the Airport has had a contract with Alcatel, a private firm, for after-hours and weekend Help Desk information technology support for Alcatel's Synchronous Optical Network, or SONET, hardware. This contract was awarded through a competitive bidding process in early 2006. In March of 2006, the Airport received Proposition J certification of such contract from the Controller but withdrew the Proposition J certification request before the Board of Supervisors when the certification request was not calendared by the Board of Supervisors. Thereafter, the Airport moved the Alcatel contract into the Airport's existing agreement with the San Francisco Terminal Equipment Company, LLC, or SFOTEC.¹ The purpose of the SFOTEC agreement between the Airport and the SFOTEC consortium is to (a) maintain, repair, operate and schedule the use of the City-owned equipment and operating systems at the International Terminal; (b) maintain, operate, and schedule the use of the International Terminal's joint use ticket counters and gates; and (c) allocate the associated costs related to the City-owned equipment and operating systems among the SFOTEC members and non-member users of the equipment. Because SFOTEC is a not-for-profit entity and not a City agency, the Airport no longer needed Proposition J certification of the Alcatel contract once the Alcatel contract had been subsumed into the existing SFOTEC agreement.

Prior to moving the Alcatel contract into the existing SFOTEC agreement, the Airport was invoiced by and made payments directly to Alcatel. Under the new arrangement, Alcatel invoices SFOTEC, which passes the invoice along to the Airport; the Airport subsequently submits payment for the invoiced amount to SFOTEC, which passes along the Airport's payment to Alcatel. Essentially, all that has changed from the prior arrangement is that management of the agreement with Alcatel has shifted from the Airport to SFOTEC; however the Airport continues to make payments to SFOTEC for the same services rendered by Alcatel as when the Airport managed the Alcatel agreement directly.

The Budget Analyst recommends that any future proposed transference of a City contract to a separate authority, so long as the contract remains paid through City funding sources, should be reviewed by the Board of Supervisors prior to final transference of such contract.

Lack of Centralized, Coordinated Communications Mechanisms for Information Technology-Specific Issues

On March 11, 2007, as a result of Federal legislation that was approved in 2005, Daylight Savings Time began three weeks earlier than it has historically. This change resulted in the need for upgrades and patches to be delivered to existing information technology systems that were programmed for the standard commencement date three weeks later. It was a change that took some effort and several City systems did not adjust appropriately and a number of calendars were off by one or two hours.

On Friday, March 9, at 5:20 p.m., the Director of Telecommunications and Information Services, sent a memorandum to the heads of all City departments, advising them that there might be time inconsistencies with their calendars the week of March 4 through March 10, and offering that departments could contact the Department of Telecommunications and Information Services Help Desk with any problems or requests for clarification. At a March 14 meeting of information technology personnel at City Hall, the Department of Telecommunications and Information Services memorandum was discussed during the agenda item on the March 11 change in Daylight Savings Time. What was clear from the discussion at the March 14 meeting was that not all the appropriate information technology personnel had received the memorandum or the information that it conveyed.

Several departments concurred with the idea that a message posting board and/or email group for information technology managers across City departments would be a useful and helpful resource, for (a) one-time issues that arise, such as the March 11 Daylight Savings Time switch; (b) ongoing City-wide information technology systems trouble-shooting; and, (c) best practices sharing across City departments.

A recommendation from the chair of the monthly meeting of information technology personnel at City Hall is the creation of two email groups of information technology managers and relevant staff at the Department of Telecommunications and Information Services: (a) one email group for Enterprise Fund-supported departments and (b) the other email group for General Fund-supported departments, as the issues encountered by the two types of Departments vary depending on the availability and consistency of financing for their information technology systems. These email groups could serve the following purposes: (a) forums for sharing information technology management and purchasing experiences and best practices suggestions; (b) resources for new information technology managers seeking basic advice from their peers; and, (c) resources for trouble-shooting.

Conclusion

There is no such thing as "Citywide Information Technology Processes" for the City of San Francisco. While there are information technology systems which span the City, such as the Financial Accounting Management Information System (FAMIS), nearly all information technology planning and purchasing processes are driven by either (a) the funding availability for each department, and/or (b) the necessity for different departments to utilize the same information technology systems. The City could benefit from greater centralization by requiring departments to develop strategic plans that are either information technology-specific or provide specific information technology goals in a larger departmental strategic plan. These strategic plans could stabilize the information technology functionality and purchasing processes of departments at the same time they provide the City with the ability to anticipate future information technology expenses and ensure that departments are maintaining and developing the information technology systems in accordance with their strategic plans.

The City could further benefit from thoughtful centralization of enterprise license agreements. Additionally, the City could realize some efficiencies from new mechanisms for inter-departmental communications.

Recommendations

The Chair of the Committee on Information Technology should:

1.1 Request each City department to develop an information technology-specific strategic plan which provides specific, quantifiable goals within a timeline that the department can check against actual outcomes.

1.2 Develop incentives that guide each department to re-visit its strategic plan as a means of ensuring achievement of strategic plan goals.

1.3 Create communications tools for information technology managers to communicate more effectively with each other.

1.4 Develop and recommend to the Board of Supervisors a protocol that requires Board of Supervisors review and approval of all City information technology contracts funded with City monies prior to transfer to a separate authority.

The Director of the Department of Telecommunications and Information Services should:

1.5 Work with the Purchaser and Controller to develop procedures to track City department purchasing requests against their long-term information technology goals in order to ensure that purchasing requests are not only needs-appropriate but also goals-appropriate.

1.6 Develop a process to continually solicit feedback from City departments in order to determine the most-appropriate technological offerings of any enterprise license agreement and then negotiate lower license costs by aggregating all City departments' total information technology needs.

The Office of Contract Administration should:

1.7 Review the Office of Contract Administration's procurement policies in order to ensure that departments have the appropriate information to make information technology procurement decisions and that processes are applied consistently across all departments.

Costs and Benefits

Implementation of these recommendations would increase City departments' efficiency in planning for and purchasing information technology equipment and applications. Potential cost savings could be achieved through more effective purchasing procedures including citywide software licenses.

1 SFOTEC is a consortium of 21 airlines with regularly scheduled International operations at the International Terminal.