What should a public-area mobile robot (PMR) be aware of as it navigates?
A 2022 study of public-area mobile robots (PMRs) in four US cities illustrated a situational awareness shortcoming with the PMRs that were used. In Detroit, the sidewalk delivery robots [1] studied were “remotely operated at all times … [and] may have line of sight issues with being able to clearly see traffic and crossing signals.” [2]
The result for this trial was that some intersections in the Detroit portion of the study were avoided, and the PMRs re-routed. In other words, the operational design domain (ODD) was constrained by its PMRs’ ability (or their teleoperators’ ability) to “clearly see” across an 8-lane intersection, approximately 26.4 meters wide [3].
Even though the PMRs’ ODD was constrained to “mild weather conditions,” the integration between the robots’ visual sensors, the teleoperators’ display systems, and the teleoperators’ procedural guide was deemed inadequate for the crosswalk shown in Figure 1. From the report, it is difficult to assess where the shortfall occurred since there is no technical reason that currently available sensors, telecommunications, and display systems could not support a visual (teleoperator) understanding of the crosswalk in clear conditions.
Figure 1: Top: an image of an avoided crosswalk from the original Urbanism Next report (p.15);
Middle: the same view from a much wider angle (current Google maps);
Bottom: the same crosswalk as viewable by a PMR prior to entering.
A FULLER REQUIREMENT
This observation requires us to pose a much more general question: How much should a PMR and its human overseer or teleoperator be able to see and hear within its operational design domain? This requirement includes cameras, lidar, mic, radar, sonar, emergency signals, police hand signals, anything or anyone approaching from the side or behind, and any other safety- or navigation-related information necessary for a PMR to operate safely among other pathway users. This requirement will be necessarily fitted according to the ODD, speed, and task of the PMR. This requirement would also apply within public spaces or pathways that may eventually permit PMRs to operate, such as sidewalks, bike lanes, airport concourses, hospital corridors, pedestrian plazas, etc.
Figure 2: Typical teleoperator setups. Note the center forward camera view and left and right cameras. Far less attention is given to a rear camera, leaving bystanders and the PMR open to critical risks.
A GENERAL APPROACH
To ensure a minimum level of PMR safety and reliability, a response to this requirement is that any PMR, given its permitted ODD speed, must have sufficient sensory systems to enable its ADS and its remote overseer or teleoperator to be able to see and hear at least as well as a human pedestrian, cyclist, or other machine user with normal (including corrected) human vision and hearing in the same ODD circumstance. If that requirement is satisfied, a PMR thus equipped would be able to negotiate the crosswalk in Figure 1 in the stated ODD conditions (daylight and mild weather conditions). To ensure sufficient time to cross, the PMR would wait for a fresh walk signal to cross unless it has access to SPaT [4] data for guidance at a given intersection.
An ISO draft technical standard under development requires a PMR to have sufficient sensory capability (suitable to its ODD, speed, and task) within a full 360° surround and without blind spots and that its ADS, teleoperator, or combination within a 360° situation assessment is sufficient to:
Detect objects and events appearing in the forward direction of travel in order to evaluate threats to its journey plan sufficiently far in advance (see Table 1) to access any perceivable threat to that plan. (Note. A critical example is that a PMR about to commence a road crossing must be able to assess the entire crossing distance, including the likelihood of any traffic signal changes and the availability of a way to exit the roadway when it reaches the opposite side).
Anticipate objects and events in either lateral view (right and left relative to the direction of travel) to be aware of any threats and anticipate the availability of navigation space if it becomes necessary to divert laterally from its optimal, planned path.
Assess objects appearing and events unfolding in the reverse direction of travel to evaluate threats to its journey plan and to remain aware of spatial resources if it must reverse course.
Table 1: minimum 20-sec sensory distance for a PMR in meters (ODD speed dependent)
PMR speed, kph | 3.0 | 4.0 | 6.0 | 10.0 | 20.0 | 30.0 | 40.0 |
PMR speed, m/s | 0.8 | 1.1 | 1.7 | 2.8 | 5.6 | 8.3 | 11.1 |
Awareness distance in meters after 20 sec | 16.7 | 22.2 | 33.3 | 55.6 | 111.1 | 166.7 | 222.2 |
Sensory and decision resources are predominantly expended forward in any navigation system with a dominant, forward direction of movement. For example, the human driver of a motor vehicle is advised to continuously scan the forward horizon of the roadway for “20-30 secs ahead” while looking to the sides or checking rear-view mirrors more or less frequently, depending on immediate circumstances [5]. Similar advice is appropriate for a PMR on a narrow pathway such as a sidewalk. However, if a security PMR is patrolling a plaza or an autonomous wheelchair PMR is transporting passengers in an airport terminal, lateral and rear awareness increase in importance and intensity of assessment.
For this reason, the draft ISO standard sets the sensory distance of the PMR to be a minimum of 20 seconds in all directions (360°). This minimum of 20 seconds indicates distances that depend on the speed of the PMR (Table 1). The ADS or teleoperator awareness distance is dependent on the ODD and task.
The implications are:
A PMR able to operate in both a bike lane and on the sidewalk requires its ADS/teleoperator to deploy two appropriate awareness distances since different ODD speeds are expected. A PMR would likely always deploy the greater sensory distance, even if the additional distance is redundant.
A PMR’s ADS/teleoperator would increase its lateral and rearward awareness distances considerably when it navigates in an area more open than a narrow pathway. This would especially be true while navigating a road crossing, which is among the riskiest instances of urban navigation.
A PMR’s ADS/teleoperator would increase its rearward awareness as it navigates in areas more crowded with bystanders. This is to ensure PMR awareness of bystanders trying to pass, alert distracted bystanders who may crash, anticipate potential vandalism, etc.
NEXT STEPS
ISO-4448 draft technical standard is being developed by the Urban Robotics Foundation (URF) under the leadership of the author and in collaboration with ISO Working Group 19. It is expected to be published in stages beginning later this year.
Members of URF are invited to review and comment on preliminary versions of the standard, and participate in working sessions to help shape the final version that is submitted for ISO committee review and approval. Please contact bern [@] urbanroboticsfoundation.org for more information or join us today!
*********************************
[1] “Sidewalk Delivery Robots,” known in US State legislation as “Personal Delivery Devices,” are one type of “public-area mobile robot.” A PMR operates in public spaces among non-involved human bystanders such as pedestrians, cyclists, micromobility users, and motor vehicles. Other types of PMRs include robots for security, maintenance, hospitality, etc., that also operate in public spaces among non-involved bystanders.
[2] Howell, A., Steckler, B., Larco, N, (2022) Piloting Sidewalk Delivery Robots in Pittsburgh, Miami-Dade County, Detroit, and San José, Urbanism Next Center. https://knightfoundation.org/reports/piloting-sidewalk-delivery-robots-in-pittsburgh-miami-dade-county-detroit-and-san-jose-knight-autonomous-vehicle-initiative/
[3] The average standard US urban lane width is 3.3m https://safety.fhwa.dot.gov/geometric/pubs/mitigationstrategies/chapter3/3_lanewidth.cfm
[4] Signal Phase and Timing, SAE J2735
תגובות