Evaluation Measures: Pros and Cons of Ridership Estimates and Accessibility Indicators  

Trillium recently conducted some research for a client interested in identifying tools for transit project evaluation. This work involved surveying available tools and analyzing their strengths, weaknesses, and potential usefulness to the client.

Trillium researched an emerging set of tools for measuring travelers’ access to destinations (“accessibility to opportunity”). Accessibility to opportunity has been measured for years. New tools and available computing power enable faster calculations and more sophisticated methods.

Travel demand modeling and ridership forecasting can also be used to assess the value and usefulness of transit projects. As part of the research, I compared what we can draw from ridership forecasts and access indicators, which is summarized in this blog post.

What are accessibility indicators?

Accessibility indicators measure how well the transit network connects households to opportunities, including employment, amenities, and activities. It does not produce an estimate of how many people will use a transit line, focusing instead on the central question of how useful a transit service is at connecting households with opportunities. Since “access to jobs, education, healthcare, and other essential services may be regarded as the primary purpose of transportation,” [1] accessibility indicators have been proposed as an alternative or supplement to ridership forecasts.

Current federal accessibility measures, such as looking at the number of households within 45 minutes of an employment center or the number of households in a buffer around a station, are very limited in the applicable information they deliver. Substituting ridership estimates with more comprehensive accessibility indicators has the potential to encourage agencies to adopt new standards and to focus on directly determining what makes a transit network useful.

Advantages of Accessibility to Opportunity

  • Accessibility measures offer certain facts about a transit network. There are few or no weights that can skew results or be influenced by bias.
  • Accessibility measures reveal advantages to all people in the network, accounting for incremental advantages to current system users.
  • Accessibility measures offer equity impacts by overlaying effectiveness of transit with equity maps.
  • A growing movement of transit advocates suggest that accessibility is the intention and utility of transit, and that therefore increasing accessibility is a primary indicator of quality and will lead to greater use.

Limitations of Accessibility Measures

  • Accessibility measures do not estimate the popularity of a service or the farebox recovery potential.
  • Accessibility measures offer no indication of the population that is riding transit, and therefore this level of equity analysis is not available. Equity measures are essentially limited to map overlays.
  • Accessibility measures do include forecasts of private vehicle travel. Therefore, they are not useful for calculating forecasted congestion, the implications of congestion on transit service, or Vehicle Miles Traveled (VMT) reduction.
  • Accessibility measures rely on demographic and land use information. If they are run to determine accessibility in the future, they depend on population, demographic, and land use forecasts that tend to only be reliable in the absence of major unexpected events.

Ridership Forecasts

Advantages of ridership forecasts

  • Ridership is currently used in the Federal Transit Administration funding evaluation.
  • Ridership can measure cost effectiveness and can serve as a variable for projecting farebox recovery.
  • Ridership calculated through models that include road travel can provide information on mode shift, congestion, total VMT, and emissions.
  • Ridership numbers are easy to communicate and easily understood by policy makers and the public.
  • Transit ridership can serve as a proxy for how useful people find the transit service within the market of transportation options.
  • Ridership forecasts that include demographic breakdowns allow for project specific equity analysis.
  • Forecasts usually include total motor vehicle throughput. This offers some important additional data.
  • Ridership can be compared with total travel in the region (e.g. is ridership higher because there are more total trips or because people switched away from cars?)
  • Models include changing congestion and may account for transit delays caused by congestion.

Limitations of ridership forecasts

  • Ridership models include many variables and weights in their algorithms. These can be difficult to review and test for accuracy. Also, simplifying these models can risk reducing accuracy. [2]
  • Ridership numbers have been found to be inaccurate in the past. Those who have demonstrated a gap in model performance blame it on several factors, including [3]:
    • inaccurate inputs (such as population and land use predictions),
    • inaccurate variables, and
    • political systems that assign funding in a way that encourages intentional or unintentional manipulation.
  • Though ridership estimates are easy to communicate and present, the assumptions used in generating those estimates may be difficult to clearly present and explain.
  • Although new, more sophisticated models and shifting criteria at the FTA have improved model performance, not enough time has passed to determine to what extent newer models are more accurate and consistent.
  • Forecasts do not account for the benefits experienced by current transit riders for current trips [4](e.g. If someone has a 60-minute commute to work and a project reduces that to 45 minutes, and ridership forecast would not demonstrate that benefit, it would only show how such a service may cause this person to make more trips by transit or may inspire other people to take trips by transit). Measuring potential travel time reductions and access improvements for current riders is part of equity analysis. Longer travel times for transit dependent riders or disadvantaged groups unjustly wastes their time.
  • Equity analysis is limited by the complexity of the population in the model and the statistical significance of a given group.
  • Certain studies suggest that project planners and promoters are prone to choosing values in their models that inflate ridership numbers, whether intentionally or by accident.

Despite being repeatedly overlooked in the past [5], questions of accessibility are becoming more prominent in the world of public transit and will likely become a more common component of project evaluation at the regional, state, and federal levels [6]. Shifting away from the focus on ridership forecasts to instead developing more sophisticated accessibility indicators has the potential to allow transit agencies to more effectively evaluate the usefulness of their systems and to make substantiated decisions about future policies and changes.

Further reading:

[1] Governors’ Institute on Community Design (2017) The Why and How of Measuring Access to Opportunity: A Guide to Performance Management.

[2] See Andersson, M., Brundell-Freij, K. and Eliasson, J. (2017) ‘Validation of aggregate reference forecasts for passenger transport’, Transportation Research Part A, 96, pp. 101–118 for discussion of the effects of cross-sectional data.

[3] See Flyvbjerg, B. (2007) ‘Cost Overruns and Demand Shortfalls in Urban Rail and Other Infrastructure’, Transportation Planning and Technology, 30(1), pp. 9–30; Pickrell, D. H. (1992) ‘A Desire Named Streetcar: Fantasy and Fact in Rail Transit Planning’, Journal of the American Planning Association, 58(2); Voulgaris, C. T. (2017) Crystal Balls and Black Boxes: Optimism Bias in Ridership and Cost Forecasts for New Starts Rapid Transit Projects. University of California Los Angeles.

[4] See Voulgaris, C. T. (2017) Crystal Balls and Black Boxes: Optimism Bias in Ridership and Cost Forecasts for New Starts Rapid Transit Projects. University of California Los Angeles for discussion of how the FTA changes its evaluation criteria, notably with the Transportation System User Benefits.

[5] Litman, T. (2017) Accessibility for Transportation Planning: Measuring People’s Ability to Reach Desired Goods and Activities. Victoria Transport Policy Institute.

[6] Governors’ Institute on Community Design (2017) The Why and How of Measuring Access to Opportunity: A Guide to Performance Management.