Grantee	Internet DNS Beijing Engineering Research Center, ZDNS
Project Title	RPKI Monitor and Visualizer for Detecting and Alerting for RPKI Errors
Amount Awarded	30,000 USD
Dates covered by this report:	2018-09-20 to 2019-09-20
Report submission date	2019-09-22
Economies where project was implemented	China
Project leader name	Di Ma
Project Team	Daiming Li [email protected] Hui Zou [email protected] Qing Shao [email protected]

Project Summary

This project implements an RPKI security mechanism that detects and counters adverse actions in the RPKI, which helps mitigate risks to global routing system. The mechanism is implemented by two components: the monitor, which detects erroneous or malicious RPKI changes, and the visualizer, which displays graphically the validation process passed to it by the validator and the alert information issued by the monitor. The project began in September 2018 and ended in September 2019.

Table of contents

• Project factsheet information
• Background and justification
• Project Implementation
• Project Evaluation
• Gender Equality and Inclusion
• Project Communication Strategy
• Recommendations and Use of Findings
• Bibliography

Background and Justification

Routing across the Internet is based on Gateway Protocol (BGP). BGP announces IP prefixes to enable inter-domain routing between Autonomous Systems (ASes). One major problem of BGP is its lack of verifiable information exchange. Several prominent incidents[1][2] highlighted the consequences: an AS incorrectly claims to originate (i.e., claims to be the destination for) an IP prefix and thereby redirects traffic, which not only may lead to traffic interruption, but can be used to intercept and tap data streams.[3]

The Resource Public Key Infrastructure (RPKI) is a security infrastructure built to complement inter-domain routing. It has been standardized by the IETF (RFCs 6480-6493) and adopted by the Regional Internet Registries (RIRs). It is gradually being rolled out by individual network operators. The purpose of the RPKI is to provide a trusted mapping from an IP prefix to a set of autonomous systems (ASes) that are authorized to originate this prefix in inter-domain routing. Such information helps to determine whether a route is valid, which can, in turn, determine the routes selected in BGP. A successfully deployed RPKI origin validation would have immediately identified the prefix hijacks referenced above – a rigorous route rejection of the invalid updates would have prevented the incidents entirely.

An entity holding a certificate capable of signing other certificates is called a Certification Authority (CA). The certificates issued under the RPKI are called resource certificates; they authenticate the holdings of INRs. Certification of resources follows the hierarchy of Internet resource allocation: The RIRs (e.g., APNIC) issue certificates for LIRs (e.g., ISPs), and LIRs for end users or other LIRs. ROAs (Route Origin Authorization) objects are issued under CA certificates and bind the mapping of an IP prefix to an origin AS. Certificates are revoked using a certificate revocation list (CRL), which are signed objects that are periodically published by the CA. A manifest is issued by a CA and contains a list of all objects that CA has published at a particular publication point. All certificates and signed objects are published in a distributed, openly accessible repository and can be downloaded by RPs (Relying Parties). (In the RPKI, the RPs are primarily ISPs.) Each RP (ISP) caches the RPKI data locally, validates the data, and uses the results of the validation process as input to routing policies of its BGP routers to enable them to verify the authenticity of routing announcements.
However, the security benefits provided by the RPKI also shift BGP’s traditional “default accept” policies to a new “default deny” mode: to prevent (sub) prefix hijacks, routers should accept routes only if authorized by the RPKI, and discard (or at least de-preference) all other routes by default. Meanwhile, the RPKI’s hierarchical architecture enables centralized authorities (e.g., RIRs) to unilaterally take down any IP prefixes under their control. Errors by or attacks against a CA or repository manager can adversely affect the Internet Number Resources (INRs) associated with that CA and/or its subordinate CAs. Errors by RIRs or other CAs at the top tiers of the RPKI could wreak havoc on Internet routing if the RPKI is widely adopted, i.e., if most network operators use the RPKI for origin validation.

“Adverse Actions by a Certification Authority (CA) or Repository Manager in the Resource Public Key Infrastructure (RPKI)” [4] describes a threat model for the RPKI. It describes a wide range of mistakes by or attacks against CAs and independent repository managers that can adversely affect the INRs associated with that CA or its subordinate CAs. The analysis is based on examination of the data items in the RPKI repository and is performed from the perspective of an affected INR holder. We will use the threat model defined by this document (a product of the IETF SIDR Working Group) as the basis for the monitoring and adverse event detection mechanisms in our system. “On the risk of misbehaving RPKI authorities” [5] showed that RPKI architecture creates technical means for powerful authorities to unilaterally revoke IP address allocations, and such means could be exploited by abusive authorities or governments to settle disputes or block undesirable content.

Here are two cases, for simplicity's sake, about how CAs might make inappropriate changes to the RPKI, either accidentally or deliberately, and thus revoke authorization for IP prefixes under their control:

Case 1: An CA revokes and/or reissues a certificate for any subordinate INR holder, causing the targeted INR(s) to become unroutable. This action could be applied not only to the CA's subordinate INR holders directly, but also to operators that receive INRs as sub-allocations from any of the CA's subordinate INR holders. This sort of action may have more severe consequences when the CA is at the top layers of the RPKI (e.g., an RIR). Consider a particular sub-case: The IANA, as an obvious candidate to be the default trust anchor for the whole RPKI system. If IANA revoked and reissued a certificate for any RIR to remove targeted resources of an INR holder, the result would have the same effect as compelling the RIR in question to revoke the certificate for the INR holder.

Case 2: Two or more CAs accidentally (or maliciously) issue conflicting certificates for the same set of INRs, which may lead to inconsistent and conflicting assertions about to whom the specific INRs have been allocated. Having a certification hierarchy with multiple equally trusted roots (i.e. RIRs) allows different organizations to make such conflicting assertions. For example, if an RIR erroneously increased the INRs of its self-signed certificate, this action could impinge on INR holders in other regions and thus affect reachability to another RIR's resources. The potential for this sort of misbehavior has just increased dramatically. In March 2016 the RIRs published a brief note (“RPKI Multiple ‘All Resources’ Trust Anchors Applicability Statement”[6)]) stating that each of them will hold the 0/0 IPv4 allocation, instead of IANA, contrary to the RPKI certificate policy. Thus any RIR could issue a certificate for any portion of IPv4 address space, and existing RPKI mechanisms would fail to detect this inappropriate behavior. This is a worrisome example of how top-tier RPKI CAs (e.g., RIRs) may act in ways that could adversely affect Internet routing. In this case the RIRs, uncertain of their ability to effect cross-RIR INR transfers as required by the RPKI architecture, have elected to lessen a safeguard built into the RPKI design, without input from the Internet standards community that developed the RPKI specifications.

Our corporation, ZDNS is the leading expert infrastructure service provider in China. One important service branch of ZDNS is IP Address Management service and relevant IPAM and DHCP devices. In China, ZDNS now has a 30% market share of DDI (DNS, DHCP, IPAM) devices, ranking the first. ZDNS's main customers include Sinopec, State Gird and other very large-scale Chinese enterprises. The safe usage of IP address resources is an important requirement of these corporations with regard to Internet Infrastructure. The advent of the RPKI and corresponding services will thus become a significant dimension of services provided to them by ZDNS. ZDNS Labs focuses on the research topics that how to make these corporations benefit from RPKI deployment.

Working to improve the robustness of the RPKI is a priority in our (internally-funded) research activities. Preliminary work on the topics described in this project was launched by ZDNS Labs two years ago. ZDNS staff have contributed to two RFCs in IETF SIDR WG[4][18], a WG draft [7] in IETF SIDROPS WG and have written a technical report[8] in Chinese, commissioned by CCSA*.

* China Communications Standards Association (hereafter referred to as CCSA) is a non-profit legal person organization established by enterprises and institutes in China for carrying out standardization activities in the field of Information and Communications Technology (ICT) across China.

Project Implementation

The Resource Publication Infrastructure (RPKI) is an effort by the IETF to secure the inter-domain routing system. It includes a formally verifiable way of identifying each entity that is a legitimate holder of each portion of the IP address space and Autonomous System Number (ASN) space. The RPKI relies on CAs (Certification Authorities) organized in a hierarchical structure aligned to the existing INR (Internet Number Resources) allocation hierarchy. Each INR allocation requires a corresponding resource certificate issued by the appropriate CA to attest to the allocation. Additionally, a ROA (Route Origin Authorization) is signed by the resource holder, binding the mapping of an IP prefix to an origin AS number. Each ROA can be validated using the certificate associated with the resource holder through the RPKI. All RPKI certificates and signed objects can be downloaded and validated using open source RP (Relying Party) validation software. Undoubtedly, the RPKI brings security benefits to the Internet routing system. However, it also enables centralized authorities that allocate INRs (e.g., Regional Internet Registries) to make inappropriate changes to the RPKI, either accidentally or deliberately, and even revoke authorization (effectively take down) IP prefixes under their control.

This project implements an RPKI security mechanism that mitigates risks to global routing, in the face of errors by or attacks upon RPKI authorities. The mechanism helps detect and counter adverse actions that result from misconfigured or compromised RPKI CA, or CAs that have been compelled to misbehave. The mechanism offers a distributed, stakeholder-based counter to the power imbalances that arise from the RPKI’s hierarchic system (which parallels the existing INR allocation hierarchy). The proposed mechanism detects adverse actions in the RPKI and alerts INR holders to these actions that adversely affect their holdings, so that errors can be quickly fixed. The mechanism also enables each ISP to decide whether to accept or defer accepting RPKI database changes that appear to be adverse. This is a decentralized approach to mitigating the impact of such actions that is consistent with the decentralized operational model of the Internet. The mechanism is implemented by two components at each RP:

1. The Monitor. Based on the output data of the existing RP validation software RPSTIR, the Monitor detects the adverse effects on INR holders which were caused by a CA’s (accidental or deliberate) misbehavior or attacks on a CA or a repository. It then issues alerts to the affected parties to remedy the error. Operating on a public service basis, or operated by each INR holder on its own behalf, the Monitor will focus on changes that appear to be adverse in the RPKI (e.g., certificate reissuance with a smaller INR set, corresponding certificate revocations, and multiple certificates with conflicting INRs), and provides suggestions to assist an RP in deciding how to respond to these suspicious changes.

2. The Visualizer. It displays graphically the model (including RPKI object contents as well as the relation between the objects) passed to it by the validator and the alert information issued by the Monitor. More specifically, given a particular ROA, the Visualizer will generate a graph output of (1) its validation state (valid or invalid), (2) its authentication chain, (3) all relevant RPKI data and their relationships in this chain, (4) all (if any) validation warnings or errors by the form of error icons on the problematic element(s).

Fig. 2. Concept Design of the Visualizer

Both of these tools, by detecting and visualizing misbehavior by RPKI authorities, allow users to evaluate results of the automatic analyses and assess the relevance of these RPKI errors. With the help of these tools, we can classify detected misbehaviors of the RPKI system in such a way that the reason for the errors are revealed or – if this is not possible – that at least plausible explanations are assessed. This output will help guide RPs (typically ISPs) in deciding whether to accept the results of possible CA misbehavior, or to defer acting on such RPKI changes. If an ISP defers acting on suspicious changes, it will make use of the most recent, previously-validated RPKI data. This hysteresis provides time for an error or the results of an attack to be remedied.

This project is aimed at improving the robustness of the RPKI system and aiding Internet operators to better understand, inspect and troubleshoot the RPKI system. Finally, we expect this project will help promote more widespread adoption of the RPKI system (by reducing the potential impact of errors, attacks, etc.) and thus enhancing security for inter-domain routing architecture.

Our project achieved the following objectives:

1. Develop an RPKI Monitor to detect RPKI problems due to mistakes by or attacks against CAs and repositories, and generate alerts to the affected parties to remedy the problems. It also provides suggestions to guide RPs in deciding whether to accept or defer accepting those changes. The Monitor will implement the following functions:

• Detect adverse actions by RIRs, including suspicious certificate revocation/reissuance, duplicate INR allocations and inter-RIR INR collision (INRs are claimed by more than one RIR).
• Provide customized service for an INR holder (RP) who provides us their INR holdings and bindings. In this service, the Monitor will verify the correctness of corresponding RPKI objects, and immediately alert the RP when a discrepancy is discovered.
• Provide general service for other RPs that does not provide their INR information. In this service, the Monitor alerts RPs of the anomaly and it provides information related to the anomaly, to assist an RP in deciding how to respond.

2. Develop an RPKI Visualizer to display graphically the validation process and involved RPKI data passed to it by the validator and the alert information issued by the Monitor. The Visualizer has three different views:

• Authentication Chain View: Users can inspect a particular ROA's validation state (valid/invalid), authentication chain, all RPKI data and their relationship involved in this chain, and all (if any) validation warnings or errors by the form of error icons on the problematic element(s).
• Publication Point View: Show the users the certificates and signed objects (ROAs, the most recent CRL and Manifest, etc.) issued by a specific CA to a publication point. This view arranges these objects in a tree structure that mirrors the hierarchical relationships between the objects.
• Detail View: Users can mouse over and click elements in the Authentication Chain View and the Publication Point View, more details then will be displayed in the Detail View, such as certificates' resources, validity period, subject, issuer, serial number, etc., ROAs' ASN, signing time, IP address blocks, etc.

The project followed the following methodology:

1. Core Software Development: This project begins with building and testing the existing RP validation software – RPSTIR on our server. By doing this, we acquire RPKI raw data and validation results of these data from distributed remote repositories. We segment the system into two components: the Monitor and the Visualizer, then carry out detailed solution design, by defining and designing (1) the interfaces between the existing validator, the Monitor and the Visualizer; (2) specific sub-functions and corresponding sub-modules of the Monitor and the Visualizer. Next, we step into the stage of developing and implementing: analyze the RPKI data and their validation results; compare the current and previous RPKI states and find the discrepancy; locate trouble spot(s) and issue an alert; visualize RPKI data, errors and alerts in the UI.
2. Experiment and Simulation: We designed and built a testbed, working with a series of experiments, which consist of unit testing, integration testing and system testing. Then we set up the simulation environment, and simulateed all types of RPKI data inputs, including typical RPKI errors and anomalies (e.g., unilateral certificate revocation, duplicate INR allocations and inter-RIR INR collision), to check whether the implementation achieved the expected results.
3. Pilot Program and Public Service: We launched a pilot program on a public platform, then collect and analyze data that's consumed and generated from the pilot, and improve the software based on this analysis. Finally, when the project is ready to be launched as a public service, we consolidated these efforts and transformed the pilot into a fully functional, state-of-the-art service for the Internet community: http://monitor.rpstir.net/.

Core Technology Breakthrough

There are some proposed RPKI security mechanisms to mitigate the risk that CAs make inappropriate changes to the RPKI. “From the Consent of the Routed: Improving the Transparency of the RPKI” [9] provides a mechanism proposing that before accepting modifications to the RPKI’s architecture to require any revocation of IP address space, RP should require consent from all impacted parties. However, this mechanism has several flaws: The consent is indicated using the resource holder’s existing private key, which is not owned by one who chooses to outsource its CA operations to its parent. As a result, the approach does not apply to the common case that currently most INR holders have outsourced CA management. Also, the mechanism does not address competition attacks (i.e., INRs are falsely claimed by more than one CA). Finally, the mechanism causes considerable impact to the current existing RPKI infrastructure by changing all affected CAs' operations.

“Suspenders: A Fail-safe Mechanism for the RPKI"[10] puts forward a mechanism that enables an INR holder to publish information about changes to objects it signs and publishes in the repository. This information is made available via a file that is external to the RPKI repository, so that RPs can detect erroneous or malicious changes related to these objects, and decide whether to accept or defer accepting such changes.

None of the existing RPKI security mechanisms address competition attacks (i.e., INRs are falsely claimed by more than one CA). More specifically, RFC8211 [4] defines the term "ROA competition": A newly issued ROA "competes" with a previoussly issued ROA if it contains the same set of INRs and they are not issued by the same INR holder. Malicious ROA competition may be an indication of potential routing disruption, since the AS involved in the competing ROA can perform a (sub)prefix hijack on the AS in the other ROA.

ROA Competition Model of RPKI

In the inter-domain routing system based on the RPKI, the validity of routing origin advertisement and whether the corresponding route data traffic is redirected depends on the "route origin verification" algorithm in the RPKI system and the "longest prefix match" algorithm in the routing system. Therefore, to construct the ROA competition model, it is necessary to consider both the "RPKI layer" and the "inter-domain routing system layer".

In the following cases, we assume that Alice’s CA issued ROA1 <10.1.0.0/16, AS1> and route1 (10.1.0.0/16, AS1), and Bob’s CA issued ROA2 <10.1.0.0/18, AS2> and route2 (10.1.0.0/18, AS2), as shown in TABLE I.

TABLE I. RPKI-BASED BGP ROUTING

Case 1: Timing Requirements

According to RFC8211, if ROA2 is issued earlier than ROA1, ROA2 is not a competing ROA for ROA1. However, considering that the RPKI protocols does not impose a mandatory requirement on the validity period of ROA, if Bob issues an ROA2' containing the same resources as ROA2 during the period when ROA1 has not expired and ROA2 is about to expire, then ROA2' as a newer ROA, has become a competing ROA of ROA1. The timing requirements in this definition introduce unnecessary complexity to RPKI systems.

Case 2: Competing ROAs with the same ASN

If an attacker invades Alice’s direct/indirect superior CA and wants to hijack 10.1.0.0/19 legally held by Alice (AS1) to the attacker’s own AS666, then it can take a covert approach: first, issue ROA3: <10.1.0.0/19, AS 1>, secondly, lie that it (AS666) is the neighbor of AS1 and advertises route3 (10.1.0.0/19, AS 2, AS 1). In this case, both the legal route1 and the illegal route3 are validated by the route origin verification algorithm of the RPKI layer. However, according to the longest prefix matching principle of the BGP layer, route3 can successfully redirect all traffic destinated to 10.1.0.0/19. Therefore, ROAs with longer prefixes and the same AS number may cause ROA conflicts, and such ROAs should also be classified as competing ROAs.

Case 3: Time difference attack

Assume Bob is a direct/indirect subordinate CA of Alice. The issuing time of ROA1 is t1, and the issuing time of ROA2 is t2. If t2 < t1, then in the period of (t2, t1), the state of route 2 is valid, the state of route 1 is unknown, and the inter-domain routing system accepts both routes at the same time, and does not change the destination of the data traffic; However, if t1 < t2, then during the period (t1, t2), the state of route 1 is valid and the state of route 2 is invalid, that is to say, the IP packets whose destination address is covered by 10.1.0.0/18 is supposed to be sent to AS2, but is hijacked to AS1 and is likely to be discarded by AS1 by forming a routing blackhole.

Case 4: Shorter Prefix to Longer

Alice can hijack Bob's route2 in a very covert way: First, modify the default value of maxlen in ROA1 from 16 to 19, then announce two routes: route3 (10.1.0.0/19, AS 1) and route4 (10.1.32.0/19, AS 1). Since both route3 and route4 are validated by the tampered ROA1, the data traffic with the destination address of 10.1.0.0/18 is hijacked by AS1 according to the longest prefix matching principle. Bob may not be able to find out all this in time, or even if he finds illegal route3 and route4, he may mistakenly believe that they will be validated as invalid by the RPKI system - because he thinks his ROA2 will always be valid and ROA1 will always be a harmless "competitor".

Combining the above four cases, we define generalized ROA competition, ROA conflicts and potential conflicts:

• Generalized ROA competition: If two ROAs contain overlapping IP prefixes, then the two ROAs are considered to be competing with each other. Specifically, the ROA containing the same or longer prefix is the competing one, and the ROA containing the same or shorter prefix is the one to be competed.
• ROA conflict: If two competing ROAs may result in routing failure or routing redirect that violates the IP resource holder's intention, then the two ROAs are considered to have conflict. If one of the two competing ROAs should be valid at the same time. If the failure may result in a routing failure or routing redirect that violates the IP resource holder's intention, then the two ROAs are considered to have potential conflict.

The Essence of ROA Competition and Conflict

In reality, there are two resource allocation/assignment scenarios in the "RPKI layer" that ultimately lead to ROA competition:

• A CA's resources specified in its own ROA are held by all CAs in its certificate chain and held by the five RIR TAs.
• A CA can repeatedly allocate resources to multiple subordinate CA certificates and/or repeatedly assigned them to multiple ROA EE certificates.

In the two cases that cause ROA competition, 1) is the basic rule of RPKI system, which cannot be avoided. In 2), there are legal scenes in reality, thus preserving the possibility of illegal scenes. The "viral propagation path" that leads to illegal scenarios (i.e., routing conflicts) is based on the RPKI certificate issuance and resource allocation path: it passes vertically through each CA node along the RPKI authentication chain, and may spread horizontally across each node. The risk of abuse of resource ownership and use rights by superior CAs aggravates the risk carried by the ROAs issued by the subordinate CAs. For a ROA issued by a subordinate CA, the more CA certificates that contain its IP address resources (all the superior certificates in the ROA certificate chain and TA certificates that contain 0/0 IP address resources in the RPKI system), the greater the risk of routing conflicts it bears.

Monitoring ROA Competition and Conflict

The ROA conflict (or potential conflict) caused by the combination of ROA competition and some specific conditions might bring threat and harm to the routing system, as shown in Fig. 3.

Fig. 3. Determine whether two competing ROA (potentially) conflicts with each other.

ROA competition itself is not necessarily illegal, and whether ROA conflicts and potential conflicts are really harmful is difficult to judge only by third-party monitors, unless there is manual intervention. For example, competing ROAs issued by the same CA with overlapping prefixes and different AS Numbers are often due to the use of anycast or multihoming in the routing system; Different CAs issue competing ROAs, either because of resource transfer (the RPKI system adopts the "make before break" rule, requiring the receiver to first issue the ROA corresponding to the IP address block migrated, and then the sender to revoke the old ROA), or because the CA issuing the longer-prefix ROA is a subordinate of the CA issuing the shorter-prefix ROA. The monitor does not know the details of legal resources held by each CA, so it is impossible to determine whether a certain resource competition is illegal, as well as the corresponding faulty party and the victim. What the monitor can do is inform the responsible CAs of the consequence of the ongoing ROA competition/conflict, and let their staff make the final decision.

The algorithm Detecting competition and conflict between two ROAs is called first. If any conflict happens, another algorithm needs to be called to determine what kind of conflict will occur to the corresponding routes if the ROA competition is projected into the routing system, and how to avoid such conflict by modifying the corresponding ROA. The suggestion on how to modify the ROA will also be presented in the form of SLURM file, which is based on RFC 8416. [18]

Findings

We have selected some typical cases:

Case 1: ROA Conflict

The two ROAs issued by different CA certificates in the RIPE NCC repository are shown in TABLE II.

TABLE II. TWO ROAS CONFLICT WITH EACH OTHER

	ROA 1	ROA 2
Origin AS	AS 47065	AS 3130
NotBefore	19/05/04 04:04:20	19/05/02 11:33:05
NotAfter	20/05/02 12:23:32	20/07/01 00:00:00
IP Prefixes	147.28.226.0/24 ……	147.28.0.0/16 ……

ROA1 and ROA2 are competing with each other because of prefix 147.28.226.0/24 in ROA1 and prefix 147.28.0.0/16 in ROA 2. The route corresponding to the former may sub-prefix hijack the latter route. The suggestion given by our monitor is to reissue ROA1 without 147.28.226.0/24.

However, since the CA issued ROA1 is the subordinate of the CA that issued ROA2, the co-existence of the two ROAs is considered legal by the current IETF RPKI protocols [3].

Case 2: ROA Potential Conflict

The two ROAs issued by the same CA certificate in the AfriNIC repository are shown in TABLE III.

TABLE III. TWO ROAS POTENTIALLY CONFLICT WITH EACH OTHER

	ROA 3	ROA 4
Origin AS	AS 32653	AS 32653
NotBefore	16/11/03 13:21:10	17/09/20 07:37:46
NotAfter	19/07/11 13:21:10	20/09/20 07:37:46
IP Prefixes	154.72.96.0/21 ……	154.72.96.0/20-24 ……

The holder of the prefix 154.72.96.0/20 includes the prefix in both ROA3 and ROA4, and sets the same origin ASN, different prefix lengths and maxlen. The time period 17/09/20 07:37:46 - 19/07/11 13:21:10 during which both ROAs are valid, and the time period 19/07/11 13:21:10 -20/09/20 07:37:46 during which ROA3 is invalid and ROA4 is valid, the route origination announcements corresponding to <154.72.96.0/20-24, AS 32653> are verified as valid, and the network traffic whose destination address covered by 154.72.96.0/20 would be sent to AS 32653.

In the time period 16/11/03 13:21:10 - 17/09/20 07:37:46 during which ROA3 is valid and ROA4 is invalid, the validation status of the route origination announcements corresponding to <154.72.96.0/20-24, AS 32653> would be divided into three parts by ROA3:

• valid: (154.72.96.0/21, AS 32653)
• unknown: (154.72.96.0/20, AS 32653), (154.72.104.0/21-24, AS 32653)
• invalid: (154.72.96.0/22-24, AS 32653), (154.72.100.0/22-24, AS 32653)

The invalid part of the route origination announcements may bring threats to the routing system.

RPKI Monitor Website monitor.rpstir.net

This website consists of the following three parts:

• Statistics of RPKI global repositories;
• Visualizer of the ROA certification chain construction, as well as errors and warnings (if any) during the construction, as shown in Fig. 4.;
• ROA competition monitor, as shown in Fig. 5.:
  • Use a ROA file name as input, locate any other ROAs in the RPKI global repositories that compete and conflict with it in real-time;
  • Display basic information contained in the two conflicting ROAs;
  • Display the impacts of two conflicting ROAs on the Internet routing system;
  • Display their relative positions in the RPKI repository;
  • Give a solution (or two alternative solutions) on how to re-issue the ROA(s) to avoid the conflict.

Fig. 4. ROA Authentification Chain Monitor

Fig. 5. ROA Competition Monitor

Project Evaluation

To avoid adversely affecting Internet routing, the RPKI concerns mentioned above should be quickly detected and addressed by each RP. However, while the RPKI puts the onus on an RP to mitigate risks created by the RPKI, There are several issues must be considered:

1. Existing RP software (RIPE NCC RPKI validator[11], RPSTIR[12], rpki.net RP tools[13], etc.) have not yet implemented such monitoring and remediation mechanisms, to address most of the aforementioned RPKI threats.

2. Even if these security mechanisms are implemented in RP software, they will bring additional complexity and heavy burden to the RP.

3. Different RPKI participants, in their roles as RPs, have different request for detecting functions. Each RIR, to maintain its credibility, should regularly retrieve and verify that its RPKI products have been issued correctly, and check if INRs it owns are claimed by another RIR. For an INR holder, its RP cares much more about RPKI threats to INRs held by itself rather than other Internet operators. For a country, it might prefer to monitor Internet resources associated with networks operating within its borders, in support of national security.

In the light of the above, the RPKI Monitor in our project will be an external and independent (i.e., not part of the RPKI hierarchy) mechanism, and incorporate customization capabilities to meet requirements of different kinds of users.

We believe that deployment of the RPKI has been hampered by (valid) concerns that high-tier CAs in the system (e.g., RIRs) represent points where errors or attacks can have sever adverse impacts on Internet routing. Our customers in China tell us this a concern. Discussions we have had with ISPs in other parts of the world have confirmed that this is a concern that many ISPs share. Thus we are eager to pursue the development of mechanisms that will address these concerns and promote more widespread adoption of the RPKI.

The current inspect tools [14][15][16][17] for RPKI were primarily built for experts and provide limited interfaces that most ISP staff would, find easy to use. Besides, they are not tailored specifically towards monitoring RPKI. For instance, they only offer limited inspection of non-ROA objects. Even when a “whacked” ROA is detected, it's difficult to identify the error source (say, a revoked or expired superior certificate or a manifest problem) in the authentication chain of this ROA. On the contrary, the Monitor in our solution will inform the INR holder of the apparent cause and source of the problem and guide the INR holder to the responsible CAs. The Visualizer, which is a graphical user interface, enables users to inspect the error position in detail.

To this end, ZDNS Labs has assembled an RPKI research team, comprised of several individuals with advanced degrees (PhDs and masters), most of whom have development experience in both DNSsec implementation and new gTLD backend service platform. Our corporation has set an explicit goal and expectation for this team, and has formulated missions including RPKI (RP) software development, CA and repository hosting service provision. Along with human resources and R&D (software and hardware) support, ZDNS also supports RPKI researchers' travel expenses, e.g., attending IETF meetings to participate in SIDR WG sessions since IETF 89 at London.

Taking advantage of our customer relationships, ZDNS has established research cooperation and related outreaching programs in China. Our corporation serves in a leading role, together with China Unicom and ZTE, working on a series of communication industrial standards named "Technical Requirements of Secure Resource Public Key Infrastructure (RPKI) Operations" in China. We plan to pursue the adoption of the RPKI, and the risk mitigation mechanisms we will develop under this grant, in this context.

Indicators	Baseline	Project activities related to indicator	Outputs and outcomes	Status
How do you measure project progress, linked to the your objectives and the information reported on the Implementation and Dissemination sections of this report.	Refers to the initial situation when the projects haven’t started yet, and the results and effects are not visible over the beneficiary population.	Refer to how the project has been advancing in achieving the indicator at the moment the report is presented. Please include dates.	We understand change is part of implementing a project. It is very important to document the decision making process behind changes that affect project implementation in relation with the proposal that was originally approved.	Indicate the dates when the activity was started. Is the activity ongoing or has been completed? If it has been completed add the completion dates.
ROA certification chain visualizer development and Monitor website development	Even when a “whacked” ROA is detected, it's difficult to identify the error source (say, a revoked or expired superior certificate or a manifest problem) in the authentication chain of this ROA.	Based on RPKI validator RPSTIR, the monitor and visualizer of ROA authentication chain are developed.	The Monitor will inform the INR holder of the validation process in the form of a certification chain, and the apparent cause and source of the problem. The Visualizer, which is a graphical website, enables users to inspect the error position.	2018/10/1 - 2019/4/20
ROA competition monitor development	None of the existing RPKI security mechanisms address competition attacks (i.e., INRs are falsely claimed by more than one CA).	Based on RPKI validator RPSTIR, the ROA competition monitor is developed.	For a given ROA, display other ROA(s) that compete and conflict with it, and output the impact this conflict does to the routing system, and one or two possible ROA reissue solutions.	2019/4/3 - 2019/9/1

Gender Equality and Inclusion

This project is a good example of Gender Equality and Inclusion. Women account for one-third of the project team, two out of six team members. Women member took equally important parts as we proceeded with the very task.

Among others, Daiming Li a female research assistant, as the architect of this project, is the main contributor to the core algorithm of ROA (Route Origin Authorization) competition detection and corresponding solutions, which is one of the cornerstones supporting this project.

Another female team member is Hui Zou, is responsible for part of the engineering work of this project and research activities on the RPKI.

Without those two ladies participation, this project would not have been carried out as expected.

Project Communication Strategy

The project team has been introducing what it is intended for in kinds of community events since ISIF grant application was approved. By doing so, the project team received feedback from the community to make the deliverable outcome of this project to promote and benefit the RPKI operation in practice.

Dr. Di Ma as the project PI gave a speech “Safeguarding the RPKI” at APCERT 2018 annual meeting, 2018 October in Shanghai, introducing the motivation behind this ISIF grant project and how it will be implemented as planned. Dr. Di Ma also gave a presentation on the same topic at China Internet Infrastructure Resources Conference (CNIRC 2019) held by CNNIC, June 30th 2019, in Beijing, to brief what is going on with this project. During this event Dr. Ma exchanged notes with China Internet community on RPKI promotion and its operation security issues.

Coming to the end of project period, Dr. Ma, on behalf of the project team gave a presentation “RPKI-X:  A Comprehensive RPKI Relying Party Service System” at APNIC 48 in Chiang Mai, Thailand, to brief the outcome of this project and how it is related to the RPKI public service provide by ZDNS where Dr. Ma serves as Principle Research Fellow.

In addition to those RPKI literacy and promotion activities, the project team has implemented a web interface to help the community get access to the outcome of this project. It is available at http://monitor.rpstir.net, an online toolset offering RPKI data statistics, RPKI certification chain analysis and RPKI data competition detection as well as corresponding remedy solutions.

By the virtue of the outcome this ISIF grant project provides, the team expects that the network operation groups will be reassured that RPKI operations could be well safeguarded based on some well-designed security mechanisms and tools. It will help address security concerns from decision makers of network operation groups. The team believes that the outcome will help our Internet community take the benefit that the RPKI brings to the internet routing system.

Recommendations and Use of Findings

Listed below are the anticipated users of this software developed under this project and the benefits it will provide with the help of this grant:

1. CA Operators

The RPKI’s hierarchical architecture creates powerful CAs with the technical means for taking down IP address space. The higher the CA is in the INR allocation hierarchy, the greater the power. Even benign errors by RIRs could lead to severe consequences on Internet routing if the RPKI is widely adopted.

The Monitor will regularly retrieve and check all certificates RIRs issue and (1) detect errors an RIR has made to the certificates, (2) analyze RIR self-issued certificates and note when INRs are claimed by more than one RIR. By doing these, this project can be utilized by RPs to mitigate errors at the top tiers of the RPKI. It also proposes ways to mitigate such actions when applied to INR holders (vs. an RIR).

2. Internet Resource Holders

Transparency of the RPKI data is crucial for resources holders to accept the RPKI. On the one hand, Public Key Infrastructures and thus also the RPKI are complex systems. The interplay of different RPKI objects is usually hard for INR holders to grasp. Moreover, almost all tools to inspect these objects are based on command line interfaces. The RPKI Visualizer component aims for easy access on RPKI certificates, CRLs, ROAs etc., and visualization of the links between different object types. Finally, resources holders will be given more confidence in their data.

When an INR holder has have outsourced CA and publication point management functions to an RIR (or to the ISP from which the INR was allocated), they do not have the private key for their own resource certificates. As a consequence, they are more susceptible to attacks than resource holders who perform these functions for themselves. The RPKI Monitor component will offer them equivalent protection against ROA invalidation and competition as well as other adverse actions.

3. Network Operators

At the same time of protecting against devastating attacks on inter-domain routing with BGP, the RPKI also brings potential negative effects to the BGP routing system by creating new risks that can inappropriately take IP prefixes offline. This project protects BGP against such risks. The Monitor in our solution can detect adverse changes and notify BGP speakers, so they can reject the change, and revert to previously validated INR data, until the responsible party remedies the error. Using the output of our RPKI Monitor, BGP speakers will put more trust on the RPKI ecosystem, and thus weight RPKI validation results more heavily in their routing policies.

4. Researchers

Even though the basic functionality of RPKI has been standardized within the IETF, there are still open research questions, in particular with respect to the ongoing deployment and safe operating procedures. The project will help researchers to better understand the current state of RPKI deployment as well as increase awareness of the threat model for the RPKI, and help them to find more comprehensive solutions in the future.

We recommend that IP address holders and network operators who have deployed RPKI and issued ROAs, should personally deploy or commission a third party to deploy the ROA monitor, to periodically detect ROA whack, ROA competition and conflict of all ROAs they issued. Organizations such as RIR and LIR that provide RPKI hosting services for a large number of INR holders should be equally accountable to their members for monitoring their ROAs.

Recommendations to RPKI CAs: How to Avoid ROA Competition Risks

From the above, in a pair of competing ROAs, the route validated by the competed ROA may be (partly) redirected by the route of the competing ROA, which might cause routing conflict; Meanwhile the route of the competing ROA has to assume the risk of unexpected failure caused by the invalidation of the competing ROA, i.e., the risk of potential routing conflict. Therefore, the resource competition between superior and subordinate CAs which is considered legal by the current RPKI standard would bring a lot of operational risks to the subordinate CA.

In addition, since there are five trust anchors in the RPKI that respectively contain all INR resources, the scope of illegal resource competition has been expanded for a specific CA. In the current RPKI system, in order to avoid a subordinate CA becoming a victim of this risk, its superior CAs and itself need to take preventive measures.

A superior CA should do the following when issuing CA certificates:

• Avoid allocating overlapping resources to different subordinate CAs;
• Try to avoid reassigning the resource to its ROA (EE certificate) which has been allocated to its subordinate CA. For example, in TABLE I, if Alice is Bob's superior CA, in order to avoid the risk of resource competition for Bob, while issuing a certificate containing 10.1.0.0/18 to Bob, Alice should reissue ROA 1, which contains only 10.1.64.0/18 and 10.1.128.0/17, and updates the corresponding routes. However this would result in more specific prefixes being injected into the Internet core and exacerbating the Internet routing scalability problem. Then Alice should at least ensure that her covering ROA takes effect after all subordinate competitive ROAs and expires before all subordinate competitive ROAs.

A subordinate CA, after issuing a ROA, should:

• Ensure the issued ROA is always valid, by making sure that every certificate in its certificate chain is valid and covers the IP resources the ROA contains, and does not expire in a short period of time. However, this is not realistic in current RPKI system.
• Ensure the issuance order of competing ROAs, by (1) making clear which of its ROAs will compete with the ones issued by its superior CA, then (2) ensuring that these ROAs are issued before the superior's competing ROAs, and expire after the superior's competing ROAs.
• Avoid circular dependency attacks. If the IP address of a CA's publish point is covered by the IP prefix of an ROA it issues, the CA should back up the publish point data to another publish point. To be on the safe side, store a copy of the competing ROA at the publish point where the covering ROA is located.
• Deploy the ROA competition and conflict monitor. The CA should deploy a monitor for ROAs issued by itself, to detect ROA competitions and conflicts within the RPKI global scope, and promptly alert the responsible party and request for repair. The CA should focus on monitoring TA certificates containing its IP address resources and all superior CAs on its certificate chain, as these certificates can directly issue illegal competing ROAs containing the same or longer prefix.

Bibliography

[1] Internet Society. The Amazon Route 53 BGP Hijack to Take Over Ethereum Cryptocurrency Wallets. https://www.internetsociety.org/blog/2018/04/amazons-route-53-bgp-hijack/.

[2]  Internet Society. Route Leak Causes Major Google Outage. https://www.internetsociety.org/blog/2018/11/route-leak-caused-a-major-google-outage/.

[3] K. Butler, T. Farley, P. McDaniel, and J. Rexford. A survey of BGP security issues and solutions. Proceedings of the IEEE, 2010.

[4] S. Kent and D. Ma. Adverse Actions by a Certification Authority (CA) or Repository Manager in the Resource Public Key Infrastructure (RPKI). IETF, 2017. https://tools.ietf.org/html/rfc8211.

[5] D. Cooper, E. Heilman, K. Brogle, L. Reyzin, and S. Goldberg. On the risk of misbehaving RPKI authorities. HotNets XII, 2013.

[6] A. Newton, C. Martinez-Cagnazzo. RPKI Multiple "All Resources" Trust Anchors Applicability Statement. IETF, 2017.
https://tools.ietf.org/html/draft-rir-rpki-allres-ta-app-statement-02.

[7] D. Ma and S. Kent. Requirements for Resource Public Key Infrastructure (RPKI) Relying Parties. IETF, 2019. https://tools.ietf.org/html/draft-madi-sidr-rp-04.

[8] http://www.ptsn.net.cn/article_new/show_article.php?categories_id=6f022632-d21e-d744-3572-44b1be26e595&article_id=sr_e1c7851c-cbe2-e365-09ec-56d7a420e177.

[9] E. Heilman, D. Cooper, L. Reyzin, and S. Goldberg. From the consent of the routed: Improving the transparency of the RPKI. ACM SIGCOMM, 2014.

[10] S. Kent and D. Mandelberg. Suspenders: A Fail-safe Mechanism for the RPKI. Internet Engineering Task Force (IETF), 2013.

[11] RIPE NCC. RPKI Validator.
https://www.ripe.net/manage-ips-and-asns/resource-management/certification/tools-and-resources.

[12] RPSTIR, Relying Party Security Technology for Internet Routing. http://www.rpstir.net/.

[13] rpki.net. RPKI Relying Party Tools. http://rpki.net/wiki/doc/RPKI/RP.

[14] NIST. rpki-monitor. http://rpki-monitor.antd.nist.gov/.

[15] Surfnet. RPKI dashboard. http://rpki.surfnet.nl/.

[16] LACNIC. RPKI looking glass. www.labs.lacnic.net/rpkitools/looking_glass/.

[17] RPKI spider. http://rpkispider.verisignlabs.com/.

[18] D. Ma, D. Mandelberg and T. Bruijnzeels, 2018. Simplified Local Internet Number Resource Management with the RPKI (SLURM). https://tools.ietf.org/html/rfc8416

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License