--- title: "r311: An R interface to the open311 standard" output: rmarkdown::html_vignette vignette: > %\VignetteIndexEntry{r311: An R interface to the open311 standard} %\VignetteEngine{knitr::rmarkdown} %\VignetteEncoding{UTF-8} --- # Introduction open311 is an international open-access standard for civic issue management and public service communication. The standard allows administrations to better manage citizen requests, citizens to more easily participate in administrative work, and researchers and data scientists to access data regarding public service communication. As an open standard, open311 is not a centralized API, but a framework implemented by various cities (e.g. San Francisco, CA, Chicago, IL, Cologne, DE, Turku, FI, Zurich, CH) and services (e.g. SeeClickFix, FixMyStreet). It is way past the golden age of open311 APIs and much of development in civic issue tracking has shifted to less open-access and less standardized alternatives. Many former prime examples have abandoned or severely limited their open311 endpoints. Nonetheless, open311 still constitutes a valuable source for open government data. Many cities and services still maintain an open311 service. `r311` allows the seamless management and selection of endpoints and retrieval of service and request data. It supports (but does not depend on) many popular R frameworks such as the tidyverse, `sf` and `xml2` for response formatting. `r311` is designed to be slim, both in content and dependencies. It imports only two import-less packages used for HTTP response handling. The functionality is limited to two main features: - Endpoint management - Sending requests This vignette will briefly cover these two features. # Endpoints Since open311 is an open and decentralized standard, there is not one but many endpoints that one can access. An endpoint is commonly implemented by a city administration, but can also be managed by a service provider such as FixMyStreet. Each endpoint can define multiple jurisdiction IDs, although, in practice, most endpoints only define a single jurisdiction. It can thus be difficult to manage the multitude of endpoints and jurisdictions. Efforts have been made to list open311 servers, but most of them are incomplete or outdated. `r311` offers an updated and modifiable endpoint list that defines a number of open311 implementations that are defined for use in the package. The list can be read using `o311_endpoints`. ``` r o311_endpoints() #> # A tibble: 69 × 12 #> name root docs jurisdiction key pagination limit json dialect deprecated deprecated_reason deprecated_url #> #> 1 "Annaberg-Buch… http… http… annberg-buc… FALSE TRUE 50 TRUE Mark-a… TRUE switched https://buerg… #> 2 "Bloomington, … http… bloomington… FALSE TRUE 1000 TRUE uReport FALSE #> 3 "Bonn, DE" http… http… bonn.de FALSE TRUE 100 TRUE Mark-a… FALSE #> 4 "Boston, MA" http… http… FALSE TRUE 50 TRUE Connec… FALSE #> 5 "Brookline, MA" http… http… brooklinema… FALSE TRUE 50 TRUE Connec… FALSE #> 6 "Austin, TX" http… http… FALSE TRUE 50 TRUE Connec… FALSE #> 7 "Chicago, IL" 311a… 311a… cityofchica… FALSE TRUE 50 TRUE Connec… FALSE #> 8 "Newport News,… http… http… cityofnewpo… FALSE TRUE 50 TRUE Connec… FALSE #> 9 "San Diego, CA" http… http… sandiego.gov FALSE TRUE 50 TRUE Connec… FALSE #> 10 "K\u00f6ln / C… http… stadt-koeln… FALSE TRUE 50 TRUE Mark-a… FALSE #> # ℹ 59 more rows ``` The list does neither claim comprehensiveness nor up-to-dateness. It arguably provides some of the most important and easily accessible endpoints as of 2025. However, `r311` also offers the ability to add new endpoints to `o311_endpoints` using `o311_add_endpoint`. You need to provide a name (for lookup) and a root URL (the URL used to send requests). The following code adds the open311 test server of Mecklenburg-Vorpommern, Germany. ``` r o311_add_endpoint( name = "MV Test", root = "https://klarschiff-mv.sis-schwerin.de/backoffice/citysdk/", jurisdiction = "rostock.de" ) ``` Retrieving the endpoints list again confirms that you successfully added a new row to the endpoints dataframe. ``` r nrow(o311_endpoints()) #> [1] 70 ``` You can now select the Rostock test API to the session using `o311_api`. This function matches an API using endpoint name and jurisdiction ID and attaches it to the active session. Query functions automatically detect the attached API. ``` r o311_api("MV Test") ``` After attaching an API, `o311_ok` confirms that the selected endpoint is able to handle request queries. ``` r o311_ok() #> [1] TRUE ``` As the result is `TRUE`, you can safely start receiving real request data. # Making requests After selecting an API and attaching it to the session, all \code{o311_*} functions can access it. You can now make requests. ## Services To get an overview of the available services in a jurisdiction, you can use `o311_services`, which returns a list of Rostock's administrative services. ``` r o311_services() #> # A tibble: 183 × 8 #> service_code service_name description metadata type keywords group group_id #> #> 1 127 "Ampel behindertengerecht gestalten" NA FALSE realtime idea Barrierefreiheit 11 #> 2 128 "Ampelschaltung \u00e4ndern" NA FALSE realtime idea Barrierefreiheit 11 #> 3 129 "Bordstein absenken" NA FALSE realtime idea Barrierefreiheit 11 #> 4 130 "Zugang rollstuhlgerecht gestalten" NA FALSE realtime idea Barrierefreiheit 11 #> 5 131 "Ampelschaltung \u00e4ndern" NA FALSE realtime idea Fahrradverkehr 12 #> 6 132 "Beleuchtung ver\u00e4ndern" NA FALSE realtime idea Fahrradverkehr 12 #> 7 133 "Beschilderung/Markierung \u00e4ndern" NA FALSE realtime idea Fahrradverkehr 12 #> 8 134 "Beschilderung/Markierung einf\u00fchren" NA FALSE realtime idea Fahrradverkehr 12 #> 9 135 "Bordstein absenken" NA FALSE realtime idea Fahrradverkehr 12 #> 10 136 "Fahrradst\u00e4nder aufstellen" NA FALSE realtime idea Fahrradverkehr 12 #> # ℹ 173 more rows ``` ## Requests To get data about civic issues in the city area, run `o311_requests`. ``` r o311_requests() #> Simple feature collection with 52 features and 15 fields #> Geometry type: POINT #> Dimension: XY #> Bounding box: xmin: 11.17392 ymin: 53.38346 xmax: 13.31174 ymax: 54.20873 #> Geodetic CRS: WGS 84 #> # A tibble: 52 × 16 #> service_request_id status_notes status service_code service_name description agency_responsible service_notice #> #> 1 263 recei… 30 "Sperrm\u00… "redaktion… "Abfallwirtschaft… NA #> 2 217 recei… 103 "Baum schad… "redaktion… "Forstamt" NA #> 3 272 "Reparatur beauftragt" in_pr… 4 "bauliche G… "Hier ist … "LK-LUP-Verkehr" NA #> 4 150 "Vielen Dank f\u00fcr … in_pr… 5 "Beleuchtun… "Stra\u00d… "Bauamt" NA #> 5 173 "Vorgang wurde zur Bea… in_pr… 3 "Ampel scha… "Ampel sch… "Regional-Adminis… NA #> 6 154 "Das Anliegen wird im … in_pr… 13 "Hundekot" "Die Halte… "Beschwerdemanage… NA #> 7 210 recei… 117 "Wegereinig… "redaktion… "Forstamt" NA #> 8 201 recei… 162 "Schutzh\u0… "redaktion… "Forstamt" NA #> 9 166 "Verstanden." in_pr… 3 "Ampel scha… "redaktion… "Bauamt (Amt Rost… NA #> 10 245 "Vielen Dank f\u00fcr … in_pr… 8 "Beschilder… "Zusatzzei… "Parchim - FB 3 R… NA #> # ℹ 42 more rows #> # ℹ 8 more variables: requested_datetime , updated_datetime , expected_datetime , address , #> # adress_id , media_url , zipcode , geometry ``` Using the output of `o311_services`, you can further narrow down the output of requests. Open311 defines a set of standard parameters which are implemented by all endpoints. Using the `service_code` parameter with one of the previously returned service codes, only complaints about broken traffic lights are returned. ``` r o311_requests(service_code = "3") #> Simple feature collection with 6 features and 15 fields #> Geometry type: POINT #> Dimension: XY #> Bounding box: xmin: 11.84164 ymin: 53.41644 xmax: 12.31198 ymax: 54.20656 #> Geodetic CRS: WGS 84 #> # A tibble: 6 × 16 #> service_request_id status_notes status service_code service_name description agency_responsible service_notice #> #> 1 166 "Verstanden." in_pr… 3 Ampel schad… "redaktion… "Bauamt (Amt Rost… NA #> 2 158 revie… 3 Ampel schad… "Die Ampel… "Stra\u00dfenmeis… NA #> 3 251 recei… 3 Ampel schad… "redaktion… "LK-LUP-Verkehr" NA #> 4 173 "Vorgang wurde zur Bear… in_pr… 3 Ampel schad… "Ampel sch… "Regional-Adminis… NA #> 5 250 "Vielen Dank f\u00fcr d… in_pr… 3 Ampel schad… "Ampel aus… "LK-LUP-Verkehr" NA #> 6 273 recei… 3 Ampel schad… "Die Daten… "Standardzust\u00… NA #> # ℹ 8 more variables: requested_datetime , updated_datetime , expected_datetime , address , #> # adress_id , media_url , zipcode , geometry ``` Similarly, using a `service_request_id` from the output, you can retrieve a single service request from the API. ``` r o311_request("250") #> Simple feature collection with 1 feature and 15 fields #> Geometry type: POINT #> Dimension: XY #> Bounding box: xmin: 11.86831 ymin: 53.43376 xmax: 11.86831 ymax: 53.43376 #> Geodetic CRS: WGS 84 #> # A tibble: 1 × 16 #> service_request_id status_notes status service_code service_name description agency_responsible service_notice #> #> 1 250 "Vielen Dank f\u00fcr d… in_pr… 3 Ampel schad… Ampel ausg… LK-LUP-Verkehr NA #> # ℹ 8 more variables: requested_datetime , updated_datetime , expected_datetime , address , #> # adress_id , media_url , zipcode , geometry ``` ## Bulk queries Many endpoints define a page limit meaning that responses are divided into pages. A query without parameters returns the first page. Pagination can be controlled with the `page` argument. To control pagination, the `o311_request_all` function can come in handy. It automatically iterates through pages and heuristically decides when to stop. The following example retrieves data from the first two pages, resulting in a tibble with 200 service requests. ``` r o311_api("Cologne") o311_request_all(max_pages = 2) #> Simple feature collection with 200 features and 11 fields #> Geometry type: POINT #> Dimension: XY #> Bounding box: xmin: 6.829202 ymin: 50.85319 xmax: 7.104497 ymax: 51.04662 #> Geodetic CRS: WGS 84 #> # A tibble: 200 × 12 #> service_request_id title description address_string service_name requested_datetime updated_datetime status media_url #> #> 1 1386-2025 "#1386-2… "Mehrere L… "50769 K\u00f… "Leuchtmitt… 2025-01-16T00:35:… 2025-02-15T12:2… closed "https:/… #> 2 1387-2025 "#1387-2… "Butzweile… "50829 K\u00f… "Schrott-Kf… 2025-01-16T06:13:… 2025-01-16T11:4… open "https:/… #> 3 1388-2025 "#1388-2… "Die Stra\… "50997 K\u00f… "Leuchtmitt… 2025-01-16T06:35:… 2025-02-15T12:2… closed "" #> 4 1389-2025 "#1389-2… "Dauerbaus… "50968 K\u00f… "Stra\u00df… 2025-01-16T07:04:… 2025-03-27T12:2… closed "" #> 5 1390-2025 "#1390-2… "Hier ist … "50931 K\u00f… "Defekte Ve… 2025-01-16T07:06:… 2025-02-16T09:2… closed "" #> 6 1391-2025 "#1391-2… "Schlagloc… "50672 K\u00f… "Defekte Ob… 2025-01-16T07:19:… 2025-01-17T07:5… closed "" #> 7 1394-2025 "#1394-2… "M\u00fcll… "51061 K\u00f… "Wilder M\u… 2025-01-16T08:08:… 2025-02-15T12:2… closed "https:/… #> 8 1393-2025 "#1393-2… "Seit Mona… "50931 K\u00f… "Schrottfah… 2025-01-16T08:10:… 2025-01-16T11:4… open "https:/… #> 9 1392-2025 "#1392-2… "Schrank" "50739 K\u00f… "Wilder M\u… 2025-01-16T08:10:… 2025-02-15T12:2… closed "https:/… #> 10 1395-2025 "#1395-2… "50733 K\u00f… "Wilder M\u… 2025-01-16T08:21:… 2025-02-15T12:2… closed "https:/… #> # ℹ 190 more rows #> # ℹ 3 more variables: status_note , service_code , geometry ``` ## Non-standard parameters `r311` implicitly supports API extensions introducing custom paths and parameters. One such API is Klarschiff Rostock which is based on CitySDK. Klarschiff CitySDK defines a number of non-default paths and parameters which extend the filtering abilities of open311 requests. Available parameters can usually be found in the respective documentation (e.g. on GitHub for [Klarschiff CitySDK](https://github.com/bfpi/klarschiff-citysdk)). The following query returns the last 50 requests tagged with the "idea" keyword. ``` r o311_api("Rostock, DE") tickets <- o311_requests(keyword = "idea", max_requests = 50) tickets #> Simple feature collection with 50 features and 15 fields #> Geometry type: POINT #> Dimension: XY #> Bounding box: xmin: 12.0452 ymin: 54.06011 xmax: 12.17308 ymax: 54.17656 #> Geodetic CRS: WGS 84 #> # A tibble: 50 × 16 #> service_request_id status_notes status service_code service_name description agency_responsible service_notice #> #> 1 48005 recei… 93 "Beschilder… "redaktion… "Tiefbauamt (Unte… NA #> 2 42258 revie… 99 "Beschilder… "Oft kommt… "Tiefbauamt (Unte… NA #> 3 50762 recei… 113 "Beschilder… "redaktion… "Tiefbauamt (Unte… NA #> 4 40539 revie… 112 "Beschilder… "Sehr geeh… "Tiefbauamt (Unte… NA #> 5 45486 recei… 113 "Beschilder… "redaktion… "Tiefbauamt (Unte… NA #> 6 42022 recei… 112 "Beschilder… "Nach dem … "Tiefbauamt (Unte… NA #> 7 46210 recei… 93 "Beschilder… "redaktion… "Tiefbauamt (Unte… NA #> 8 50520 recei… 113 "Beschilder… "redaktion… "Tiefbauamt (Unte… NA #> 9 40653 "Vielen Dank f\u00fcr … in_pr… 93 "Beschilder… "Wenn ich … "Tiefbauamt (Unte… NA #> 10 42112 recei… 99 "Beschilder… "redaktion… "Tiefbauamt (Unte… NA #> # ℹ 40 more rows #> # ℹ 8 more variables: requested_datetime , updated_datetime , expected_datetime , address , #> # adress_id , media_url , zipcode , geometry ``` Some custom parameters can also alter the shape of the output. In the following example, we query just the count of total requests using the `just_count` parameter. The result is a 1×1 tibble containing a count value. ``` r o311_requests(just_count = TRUE) #> # A tibble: 1 × 1 #> value #> #> 1 3310 ``` The CitySDK extensions also offers additional URL paths which can be queried using the generic `o311_query` function. ``` r poly <- o311_query("areas") plot(poly$grenze) plot(tickets$geometry, add = TRUE, pch = 16) ``` ![](../man/figures//map-1.png) # Cleanup Endpoint data is stored persistently between sessions so that you can create your own database of open311 endpoints. This database is stored in the system's user directory as returned by `tools::R_user_dir("r311")`. To reset the database, run ``` r o311_reset_endpoints() ``` This will default back to the endpoints defined by `r311` and remove all endpoints manually added by `o311_add_endpoints`.