SF's tech shuttle pilot project is going to create some really big data, appropriately enough

By Dan Raile , written on April 11, 2014

From The News Desk

The smoke has cleared, the dust has settled. The last gasp of resistance to the San Francisco Commuter Shuttle Policy and Pilot Project was voted down last Tuesday, and now, after years of white-knuckled waiting for the Goldberg machine of city administration to run its course, the folks at the Municipal Transportation Agency can finally start regulating employee shuttles, known colloquially as ‘Google buses’.

On Monday I spoke with Carli Paine, the MTA project manager overseeing the shuttle program. She seemed relieved at being able to put the controversy and uncertainty of last week’s appeal hearing behind her, and eager to refocus on the challenges of implementing the program by its scheduled start-date in July.

The MTA already has a sprawling mandate. In 1999, Proposition E folded the Department of Traffic and Parking into the agency. In 2009, the Board of Supervisors forced the MTA to absorb the Taxicab Commission and regulate the car-for-hire industry. The agency also oversees pedestrian and cyclist concerns, in addition to operating and maintaining the SF Muni public transit system of buses, light-rail, trolley-buses and cable cars. Oh, and then there is the parallel emergence of ridesharing apps like Uber and Lyft to contend with, and carsharing services like Zipcar.

And now those responsibilities include regulating the florid ecosystem of private commuter shuttles.

The guiding philosophy of the MTA’s consolidation is efficiency – all of these functions compliment and affect one another in the real world, collectively they form the overall transit picture of the city. By housing them all under one roof, the city can provide an optimal transit experience for its citizens, the thinking goes.

“San Francisco is unique in that one department has authority over both the Muni and the streets,” Paine says. She adds that this should give the MTA an advantage in coping with the booming shuttle operations. And they'll be needing it.

“No other U.S. city has this mass of commuter shuttles. Here in the Bay, other cities [where the shuttles operate] are looking to us to see how we handle it.”

The public perception seems to be that the Pilot Program was produced by Mayor Ed Lee and the honchos atop massive Peninsular corporations from behind closed doors and out of thin air, largely in response to recent anti-shuttle protests. But that's not entirely the case.

The MTA has had its eye on handling shuttles since at least 2008, when a Strategic Analysis Report to study the impacts and benefits of such operations was initiated. That study concluded in 2011, and delineated a proposed program that looks much like the one we have now.

Another round of information gathering and policy development occurred when, in 2011, the SF County Transit Authority told the MTA that it really ought to regulate the shuttle industry. The agency received a grant from the Bay Area Climate Initiative to the tune of $750,000 to produce* a pilot program, a process which ended in the fall of 2013. The pilot program proposed is basically the one set to take effect in July, with some modifications.

[Update: MTA's Carli Paine emails me to clarify: "The grant was for a host of transportation demand management efforts across SFMTA, SFCTA, SF Dept of Environment, and Planning Dept, including the SFMTA’s development of a policy for shuttles."]

In January of this year, the MTA Board of Directors issued a resolution that authorized the MTA to establish the pilot program.

The MTA is not going it alone: the Planning Department, the Department of the Environment and the County Transit Authority are partners in the project on the public side; the 17 shuttle providers and the (currently unknown number of) employers that contract them are private partners.

But the MTA has its hands full with integrating the shuttle program into its existing transportation networks. Though America has a history of locating its industrial parks away from dense populations of workers, a large-scale, public-private, mass-transport hack like regional commuter shuttle system has little precedent.

Among the MTA's tasks for the Pilot:

It must produce and process the applications from shuttle providers to obtain permits to use Muni stops, and these applications must also include data about proposed routes, and the location and frequency of stops. The shuttle providers are required to outfit their vehicles with data tracking devices, and provide that data to the MTA, which must then analyze the data to assess impacts and benefits as well as violations.

The MTA must also develop an enforcement strategy for assessing penalties for non-compliance, and collection of those fines...and the agency must determine which Muni stops will be included to be among the approximately 200 to be included for the pilot.

It should be becoming clear that we are talking about a lot of data here.  And this new abundance of data about the shuttle operations seems a welcome development for the administators. Both Carli Paine of the MTA and Tilly Chang, the Executive Director of the SF County Transportation Authority, told me that in the past, good data about the shuttle programs was hard to come by.

“[The pilot program] is all about cooperation and consensus. Employers took some time to get comfortable with the idea of regulation,” Chang said.

Paine said that MTA’s past efforts at studying the situation was hampered because participation on the part of employers and shuttle companies was voluntary, not a high priority and voluntary.

“The biggest challenge was getting cooperation from the companies,” Paine said, “the employers said they didn’t keep the data themselves, that it resided with the charter companies. And the charter companies have non-disclosure agreements stating that they won’t share that data.”

Over the course of the Pilot, the agency is being required to analyze all that data in order to develop the permanent plan for regulating commuter shuttles to best integrate them into the city’s overall transit picture.

The data-analytics capacity is still being developed, and will be handled internally by the MTA’s IT experts.

Luckily, we live in the age Big Data, as we’re told. And San Francisco’s city government has been a big advocate of open government initiatives, which precipitated the Mayor’s Open Data Ordinance last fall. As part of that ordinance, the city hired Joy Bonaguro as its first Chief Data Officer in early March, and is in the process of hiring up department-level Data Coordinators to support her.

According the (expired) job listing for the CDO position:

The responsibilities of the Chief Data Officer are threefold: promoting transparency through open data, constructing a comprehensive strategy for citywide data usage and storage, and facilitating data-driven decision-making through sophisticated analytics.
Unfortunately, it seems that the Open Data Ordinance remains unexploded for now. When I asked Carli Paine and MTA Public Relations Officer Kristen Holland if they were working with the CDO, they admitted that they weren’t familiar with Bonaguro, or the Chief Data Officer position.

Which leads one to believe that all the data collected as part of the Pilot Program, and the subsequent analysis, will not only be handled internally by MTA. It might be buried there, too.

Of course, some plucky citizen could try requesting the information under the Sunshine Ordinance, a former San Francisco transparent-governance initiative that some say was cast on rocky soil.

A quick glance between the lines suggests that part of the public-private partnership agreement was that this data not be public-facing. And given recent events, there is a case to be made for security concerns related to making that data available.

To some it may seem paradoxical or hypocritical that certain Peninsula-based corporations with shuttle operations would be so stingy when it comes to sharing this little slice of their data troves. But guarding and monetizing that data is a guiding principle of their business, and old habits die hard with a vengeance, as they say.

Yet there is a case to be made for a public-facing portal for all this data. Beyond the ethics of open data, there is the fact that a comprehensive data-map of all of San Francisco’s available transit options would be useful to citizens and government alike. Since MTA hosts all this data, it could generate a multi-layered map for all to behold. There is also the possibility that providing this data to the public could result in an improved analytic capacity being generated and delivered to the MTA, lightening its load.

For now, the wheels of the bus go round and round, as do the gears of government. At least they’ve finally started sharing with each other. But it seems that the rest of us will have to wait and see, unless we get some Sunshine.

[Illustration by Hallie Bateman for Pando]