Grantee	LUMS University
Project Title	Establishment of a Carrier Neutral Software-Defined Internet Exchange (IXP) Point and Training Programs for Capacity Building in Managing IXPs
Amount Awarded	USD 30000
Dates covered by this report:	2018-11-01 to 2019-12-31
Report submission date	2020-03-04
Economies where project was implemented	Pakistan
Project leader name	Zartash Afzal Uzmi
Project Team	Ahmad Nauman [email protected] Haider Ali [email protected] Ihsan Ayyub Qazi [email protected] Fawad Raza [email protected]
Partner organization	Pakistan Telecom Authority (PTA), Higher Education Commission (HEC) of Pakistan

Project Summary

Internet exchange points (IXP) are a critical piece of the Internet infrastructure that enable ISP networks to exchange traffic with each other. The Internet has more than 300 IXPs worldwide. IXPs offer a number of benefits including cost savings, better performance, and security. Traditional IXPs do not leverage the modern networking evolution offered by Software-Defined Networking (SDN). This project aims to develop and deploy a Software-Defined IXP (SDX), using recent advances in SDN to allow operators to enable novel applications such as application-specific peering, traffic redirection through middleboxes, and inbound traffic engineering. We have already created a working prototype of SDX, comprising a route server (Quagga Application), Two (2) ISPs (2 PCs with Quagga BGP configurations), Aruba Openflow switch and Ryu Controller. Two novel peering applications have also been implemented on this SDN-based IXP. To support the management and configuration of novel peering applications, we have developed a new package to SDN-enable the “IXP Manager” which is a web-based peering portal used by 79 traditional, non-SDN based, IXPs worldwide. Our package integrates with an open-source version of a traditional IXP Manager and allows configuring SDN-based applications through the peering web portal. This retrofitted IXP Manager will not only be useful for upcoming SDN-based IXPs, particularly in developing countries, but will also be useful for the existing IXPs worldwide as they introduce SDN capabilities within their infrastructure. While the development work of the project has been completed, we will continue exploring avenues for large-scale deployment. As a first step in this direction, this project centered on Pakistan IXP; a future goal will be to use it as a testbed for full-scale deployment, testing, and evaluation. We will also continue to carry out training programs to prepare additional human resource in managing IXPs as well as in using SDN controllers. An additional future direction stemming from this project is to build a sustainable basis of discussion, collaboration, and training programs between least developed countries around SDN, IXPs, and other emerging technologies. The establishment of an SDN-based IXP stands to bring down operating costs of IXPs via automatic configuration management and dynamic policy assignment. The SDN-based IXP platform developed in this project will also uncover the empirical data highlighting the benefits of IXP, particularly in the developing world, by measuring the inter-ISP traffic volumes. We further aim (as a future goal) to study the traffic types to estimate the growth in content hosted locally, or moved over from international to local hosting. Our project will eventually be useful for persuading popular large-volume publishers (Facebook, Netflix, YouTube, etc.) and content distribution networks (CDNs) to establish their local presence within the country and peer at the IXPs in Pakistan. This project will further allow the Pakistan IXP team to right size the future IXPs (in Karachi and Lahore) and scale the one in Islamabad. The knowledge of “where” the traffic is destined and downloaded from “outside the country” will motivate additional local and international cloud service providers to get interested in hosting their platforms within the region.

Table of contents

• Project factsheet information
• Background and justification
• Project Implementation
• Project Evaluation
• Gender Equality and Inclusion
• Project Communication Strategy
• Recommendations and Use of Findings
• Bibliography

Background and Justification

Internet exchange points (IXP) are a critical piece of the Internet infrastructure that enable ISP networks to exchange traffic with each other. The Internet has more than 300 IXPs worldwide—with more than 80 in US/Canada alone—and some IXPs carry as much traffic as the Tier-1 ISPs. IXPs offer a number of benefits including cost savings on International transit costs, better performance and user experience for locally hosted content, and improved security and availability (e.g., in case of disruption in International bandwidth such as due to cuts in undersea cables). However, several less developing countries face two key challenges:

1. They either lack IXP infrastructure or have recently deployed IXPs with limited capabilities (e.g., Pakistan, the 6th most populous country, deployed their first ever IXP in Islamabad just last year), and
2. They lack expertise and human resource for operating and managing IXPs, which is essential for realizing the true potential of IXPs.

As a first step, this project centered on Pakistan IXP. However, our code, documents, and knowledge is open to other beneficiary countries to participate and benefit from this project. The ultimate goal of this proposal is to build a sustainable basis of discussion, collaboration, and training programs between least developed countries around SDN, IXPs, and other emerging technologies. The establishment of a SDN based IXP stands to bring down operating costs of IXPs via automatic configuration management and dynamic policy assignment.

The open source package developed as part of the IXP Manager will be useful for the IXPs in developing countries. This package will also be of tremendous help for the IXP operators worldwide, when they implement SDN based IXP, requiring just a minimal effort of installing our package in their IXP managers. The IXP Manager is simply a web application that provides the participating ISPs a peering portal where they can enter peering information independent of the underlying IXP infrastructure, helping IXPs and ISPs in terms of time and operating costs. We intend to continue holding the training programs on how to use the IXP manager and our developed package in future.

The SDN-based IXP platform developed in this project also uncovers the empirical data highlighting the benefits of IXP, particularly in the developing world, by measuring the inter-ISP traffic volumes. We further aim (as a future goal) to study the traffic types to estimate the growth in content hosted locally, or moved over from international to local hosting. Our project will eventually be useful for persuading popular large-volume publishers (Facebook, Netflix, YouTube, etc.) and content distribution networks (CDNs) to establish their local presence within the country and peer at the IXPs in Pakistan. This project will further allow the Pakistan IXP team to right size the future IXPs (in Karachi and Lahore) and scale the one in Islamabad. The knowledge of “where” the traffic is destined and downloaded from “outside the country” will motivate additional local and international cloud service providers to get interested in hosting their platforms within the region.

Pakistan is the sixth largest country in the world with over 200M population and an internet penetration just over 20%. The country is served by a few tens of internet service providers, small and large, nation-wide and regional. Unfortunately, the IXP initiative is relatively new in Pakistan with the first IXP established only recently (in 2017). The lead researcher of this project serves on the IXP board and the project team played a key role in making the IXP possible. Despite this, the inter-provider connectivity structure in the country is far from desirable. This project not only has provided this platform to easily interconnect and manage ISPs but will also, in future, pave ways for global content and application service providers and CDNs to peer at the IXP. This future direction can be driven by the current outcome of this project which makes it possible to have an easy access to the inter-carrier traffic volumes observed at the IXP, easily measurable by the use of SDN.

The project team is actively engaged with the IXP operations and this project will only strengthen the relationship between the academic community with the service provider community. Another outcome from the project was the training of human resource in understanding inter-ISP connectivity at a large scale. This, in the long run, will help the general service provider body to run their operations more efficiently and without running into routing issues stemming from lack of operational expertise (case in point: the inadvertent blockage of YouTube for most of the world for a couple of hours, back in 2008 due to a BGP misconfiguration error by a large service provider from Pakistan).

The expertise of the project team in SDN, IXP, BGP and measurements (as they had published papers on these topics in top conferences) and a vision of the positive impact this R&D would create in developing countries served as the motivation for the team throughout the project.

In terms of previous work, there has been no such attempt in the past for the SDN based IXP in developing countries like Pakistan with access to the actual deployment. There are few efforts made in the world like the SDN based IXP deployed at Toulouse IXP, France. Our work will help all the IXPs around the world to solve operating problems faced at IXP.

Initially, it wasn't decided how much of automation would we include in the design and development. The thought was that if we automate the process for peering between ISPs and applications of SDN with a web based user interface then it would not only ease the job of operator at IXP but would also make the ISPs independent of the IXP operations. The path we actually took, i.e., complete automation through web portal was a change in the overall design of the SDN based IXP we proposed earlier in the proposal. It may be noted that the peering process in normal IXP in Pakistan, like most other normal IXPs in the world, is manual that could benefit from automation. Our team proposed the use of open source web portal IXP Manager (which is being used by 79 IXPs worldwide to automate IXP operations). But that open source web portal was only available for a simple IXP and not for the SDN enabled one. We, therefore, developed a package in IXP manager in order to turn it into an SDN based IXP Manager. It was developed to automate the peering process and configuration of novel peering structures such as inbound traffic engineering, server load balancing, and application specific peering.

In terms of organizational support, the university freed up faculty time to complete the project. Costs associated with hiring, administration, and payroll services were handled by the university. Auditoriums for workshops and training sessions were available during the project development. Lab space and research space including other associated services such as secretarial staff services, communications, and printing, etc. were also provided by the university free of cost, as promised in the project proposal. The university also provided the project management services through its office of research without charging any overhead.

Project Implementation

Summarizing the series of steps which are carried out and explained in the below mentioned manual (starting from Introduction):

1- The architecture of IXP was designed by making an IXP using Open Virtual Switch (OVS) and three PCs (one acts as route server, other two act as ISPs).

2- Then the ryu controller was used to install flow rules in OVS which make it Software defined Internet Exchange Point (SDX)

3- After that, OVS was replaced in the architecture with the Aruba openflow switch. Switch configurations were done such that switch listens to controller on specific port. Also, we configured switch further to make control plane and data plane work independently.

4- After that, the “IXP manager” (open source IXP web based portal being used by 79 IXPs worldwide) was installed and connected to switch.

5- The architecture of the Software defined Internet Exchange Point made till now:

6- Applications like application-specific peering and inbound traffic engineering are being implemented now.

7- Now, IXP Manager is being developed to be connected to controller so that ISPs or IXP administration can install flow rules from portal to controller.

8- Further objectives or goals would be to hold training programs and do measurement studies using PERN/TEIN networks as mentioned earlier in the proposal submitted.

9- Resources are given at the end of the manual below.

SDX Architecture Using PCs

Introduction:

This document provides an introduction to a demo implementation of a software defined internet exchange point (SDX) using three desktop PCs. The demo implementation is fundamental to the switch-based implementation.

An Internet Exchange Point (IXP) on the internet refers to a location (and associated networking facilities) where different independently operated networks (sometimes also called autonomous systems, or domains) exchange traffic with each other. Creating a software defined internet exchange point could make some aspects of wide-area network management easier by giving operators direct control over packet-processing rules that match on multiple header fields and perform a variety of actions.

The setup for PC-based (demo) implementation:

Setup Summary:

• This setup requires three desktop PCs of which two will be configured as ISPs (PC1, PC2) and the third one is used as an IXP (PC3).
• Both the ISPs (PC1, PC2) are running the Quagga bgpd (BGP Daemon). Meanwhile, an OVS switch and a route server are configured on PC3 (the IXP).
• An SDN controller (Ryu) is used for passing flow rules to switch. Any other SDN controller may also be deployed but we choose to use Ryu controller for its widespread use and better support.
• Appropriate networking hardware was added to the PCs to support the architecture shown in the figure.

Configuration Summary:

• There is a BGP configuration file in each of the PCs. In PC1 and PC2, which are the ISPs in our setup, the BGP configuration file will contain information regarding that AS such as the AS number, neighbors, etc.
• Router server’s BGP configuration is in PC3 having information regarding all the peers that are connected to IXP. In this setup, only PC1 and PC2 are connected to the IXP (PC3).
• Each participant (or service provider) may further write its own policy in the SDX controller, one file per participant. All the data from these files will then be compiled together to form inbound outbound flow rules which will then be passed on to the switch by Ryu manager.

Step 1: ISP Connectivity Setup without IXP (warm up with BGP)

Configuring the ISPs and passing the BGP advertisements between them through bilateral peering (without IXP).

In this step we connect two PCs via ethernet cables, install quagga on both PCs and then start a BGP session between them. Since there is no IXP involved, hence it will be a bilateral peering. Each new PC will have to maintain a separate session, thus creating a mesh topology.

Configuration Steps:

Step1: Quagga Installation Steps on both PCs:

1. Run this script on both PCs in order to install Quagga package. $ sudo su # apt-get update # apt-get install quagga quagga-doc Once the installation is done, we need to edit the daemons file in order to configure the required daemons for our setup. # nano /etc/quagga/daemons
2. We then modify this to specify which daemons to install in the PC file, as follows: zebra=yes bgpd=yes ospfd=no ospf6d=no ripd=no ripngd=no isisd=no babeld=no
3. Save the file and quit the editor.
4. Create config files for the zebra and bgpd daemons. # cp /usr/share/doc/quagga/examples/zebra.conf.sample /etc/quagga/zebra.conf # cp /usr/share/doc/quagga/examples/bgpd.conf.sample /etc/quagga/bgpd.conf # chown quagga.quaggavty /etc/quagga/*.conf # chmod 640 /etc/quagga/*.conf
5. Start Quagga: # /etc/init.d/quagga start
6. Set up environment variables. Edit the /etc/bash.bashrc file:
   # nano /etc/bash.bashrc
7. Add the following line at the end of the file:
   export VTYSH_PAGER=more
8. Save the file and quit the editor. Then, edit the /etc/environment file:
   # nano /etc/environment
9. Then add the following line to the end of the file:
   VTYSH_PAGER=more
10. Save the file and quit the editor.
11. Start the Quagga shell with the command vtysh:
    # vtysh

BGP Configuration:

• ISP1

PC1 is configured as ISP1 and the IP assigned to it is 11.0.0.1

R1(config)#interface eth1
R1(config-if)#ip address 11.0.0.1/24
R1(config-if)#no shutdown
R1(config-if)#interface loopback0
R1(config-if)#ip address 1.1.1.1/24

#Now to add the neighbor’s information, use the following commands.

R1(config)#router bgp 7675
R1(config-router) #neighbor 11.0.0.2 remote-as 7676

#Now to advertise a route

R1(config-router)#network 1.1.1.0/24

• ISP2

PC2 is configured as ISP2 and the IP assigned to it is 11.0.0.2

R2(config)#interface eth1
R2(config-if)#ip address 11.0.0.2/24
R2(config-if)#no shutdown
R2(config-if)#interface loopback0
R2(config-if)#ip address 2.2.2.2/24

R2(config)#router bgp 7676
R2(config-router)#neighbor 11.0.0.1 remote-as 7675

Step 2: Configuring the IXP without route server, and connecting the ISPs through IXP layer 2 switch

In this step we add PC3 in our topology which is used as an internet exchange point. Open virtual switch (OVS) is installed on PC3 and then both other PCs are connected to this switch via ethernet cables. Here PC3 acts as a simple layer 2 switch and performs no other functionality than forwarding the packets.

OVS Installation on IXP(PC3):

#! /bin/bash

apt-get install git

Downloading OVS:

git clone https://github.com/openvswitch/ovs.git

cd ovs

Checking the master version:

git checkout remotes/origin/master

Installing Dependencies

apt install build-essential libssl1.0.0 libcap-ng-utils

sudo add-apt-repository ppa:deadsnakes/ppa

sudo apt-get update

sudo apt-get install python2.7 libnuma-dev libtool autoconf automake wget python-six libvirt-bin

sudo add-apt-repository ppa:ubuntu-toolchain-r/test

sudo apt-get update

sudo apt-get install g++-4.9 gcc-multilib

apt-get install libpcap-dev

apt-get install iperf

./boot.sh

./configure

make

sudo make install

sudo make modules_install

sudo modprobe openvswitch

lsmod | grep openvswitch

export PATH=$PATH:/usr/local/share/openvswitch/scripts

ovs-ctl start

Configuring OVS interface and adding two more interfaces in switch for connection of ISPs

ovs-vsctl add-br br0 #to add a switch named br0

# to show switch configuration

ovs-vsctl show

# to add port in br0 switch

ovs-vsctl add-port br0 eth4

# to add port in br0 switch

ovs-vsctl add-port br0 eth5

# to add port eth4 in br0 switch with ofport=1

ovs-vsctl add-port br0 eth4 -- set Interface eth4 ofport=1

# to see flow rules on switch br0

ovs-ofctl dump-flows br0

Switch’s configuration will look like this after running ovs-vsctl show command.

Step 3: Configuring the IXP with route server

In order to avoid mesh topology, we add route server on IXP that way each ISP will have to make only one connection i.e. with route server. Route server will act as a broker and will pass on information received from each ISP to every other connected ISP.

For configuring route server on IXP (PC3) we will install Quagga on that PC following the same method as mentioned in step1. Once the Quagga is installed, configuration of route server and ISPs will be done like this:

BGP Configuration:

• Route Server

In order to inform route server about its clients, route-server-client command is used.

RS(config)#interface eth0
RS(config-if)#ip address 11.0.0.4/24
RS(config-if)#no shutdown
RS(config)#router bgp 7677
RS(config-router)#neighbor 11.0.0.1 remote-as 7675

RS(config-router)#address-family ipv4 unicast
RS(config-router-af)# neighbor 11.0.0.1 activate
RS(config-router-af)# neighbor 11.0.0.1 route-server-client
RS(config-router-af)#exit
RS(config-router)#neighbor 11.0.0.2 remote-as 7676

RS(config-router)#address-family ipv4 unicast
RS(config-router-af)# neighbor 11.0.0.2 activate
RS(config-router-af)# neighbor 11.0.0.2 route-server-client
RS(config-router-af)#exit

• ISP1

In order to generate thousands of dummy prefixes, a script[4] was used, these prefixes were added in the bgpd configuration file and then were passed to route server which were then further advertised to ISP2.

R1(config)#interface eth1
R1(config-if)#ip address 11.0.0.1/24
R1(config-if)#no shutdown
R1(config-if)#interface loopback0
R1(config-if)#ip address 1.1.1.1/24

R1(config)#router bgp 7675

R1(config-router)#no bgp enforce-first-as
R1(config-router)#neighbor 11.0.0.4 remote-as 7677

#Now to advertise a route
R1(config-router)#network 1.1.1.0/24
R1(config-router)#network 1.1.2.0/24

• ISP2

R2(config)#interface eth1
R2(config-if)#ip address 11.0.0.2/24
R2(config-if)#no shutdown
R2(config-if)#interface loopback0
R2(config-if)#ip address 2.2.2.2/24
R2(config)#router bgp 7676
R1(config-router)#no bgp enforce-first-as
R1(config-router)#neighbor 11.0.0.4 remote-as 7677

Step 4: Passing flow rules to switch using RYU controller

In order to pass flow rules to switch for fileting packets based on different header fields we attach a ryu controller with switch. Flow rules are written in controller which are then passed on to the switch. Now every packet is treated based on the flow rule, and if a packet doesn’t match with any flow rule it’s forwarded to the controller and then the controller decides what to do with the packet.

RYU Installation on IXP(PC3):

Installing Dependencies

sudo apt install python-pip

pip uninstall requests

pip install requests==2.18.0

pip install rfc3986==0.3.1

pip install PyYAML==3.12

pip install stevedore==1.22.0

pip install debtcollector==1.3.0

pip install oslo.i18n

sudo pip install --upgrade setuptools

RYU Source Download

wget https://gist.githubusercontent.com/ajinkyakadam/d6ef527a2ddbfb29bc53fdaf2270c228/raw/4cb0b694055e19b2aa5a166d6328070138c6a8b0/ryusetup.sh

bash ryusetup.sh

Attaching Controller with Switch:

Following command will point software openflow switch the controller whose ip address is 11.0.0.4

sudo ovs-vsctl set-controller br0 tcp:11.0.0.4:6633

One can verify OVS settings by issuing this command

sudo ovs-vsctl show

To delete previous flow entries on switch

sudo ovs-ofctl del-flows br0

Now to pass the flow rules to switch, simply run the python code where rules are written. For example

./bin/ryu-manager ryu/app/simple_switch.py

Output on termial should look like this

To see the flow table entries on OVS switch, following command can be used

sudo ovs-ofctl dump-flows br0

Output of above command

SDX Architecture Using PCs and Aruba Switch (JL258A)

Setup Details:

Now moving on from our current setup, by replacing the OVS switch with an OpenFlow enabled Aruba switch. Since switch is not part of PC3 anymore, route server and Ryu controller (which are still running in the PC3) will have to make a separate connection with the switch via ethernet cables.

Switch Configuration:

The switch has a total of ten ports of which eight ports (1-8) are 1 GbE PoE+ and two ports(9-10) are 10 GbE SFP+. Configuration of the ports which will be used is as follows:

configure

• VLAN 1

VLAN 1 is the default VLAN that already exists in the switch. All the 10 ports are included in this VLAN. In order to be able to make telnet connection with switch in putty, we will assign an IP address to this default VLAN.

vlan1 ip address 192.168.1.1/16

• Controller

We need a controller interface so that RYU can pass flow rules to the switch.

openflow

openflow enable

controller-id 1 ip 192.168.1.2 port 6633 controller-interface vlan 1

• VLAN 2

Since we don’t want to keep our control plane packets and data plane packets in the same vlan, so we will create another VLAN as VLAN 2 in which we will add two ISPs and the route server.

For assigning IP to VLAN 2

vlan 2 ip address 10.0.0.10/24

As the number of participants in this VLAN are three, we will have to add three ethernet ports in VLAN 2 from VLAN 1.

vlan 2 untagged ethernet 6-8

• OpenFlow

For creating a new OpenFlow instance named l1

openflow instance l1

Adding the listening port for this instance

openflow instance l1 listen-port 6633

Once the instance in created, we will add the controller in it and then make it member of vlan 2.

openflow instance l1 member vlan 2

Since this switch supports OpenFlow version 1.3, we will define the OpenFlow version 1.3 for l1

openflow instance version 1.3 only

Once the instance is configured as required, we will enable it

openflow instance l1 enable

Final configuration of the switch will look as follows

Ryu Controller:

As we have defined open flow version 1.3 in the OpenFlow instance configuration on the switch, therefore we will be using example_switch_13 file from ryu directory. For now, we will pass flow rules to the switch which will make it perform like a simple switching hub having following functionalities:

• Learns the MAC address of the host connected to a port and retains it in the MAC address table.
• When receiving packets addressed to a host already learned, transfers them to the port connected to the host.
• When receiving packets addressed to an unknown host, performs flooding.

In OpenFlow version 1.3, there are a total of three tables in which flow rules are written. Details of these tables can be found out by running this command:

show openflow instance l1 flow-table

By default, RYU tries to add flow rules in the start table of the switch, which the switch doesn’t permit. By running the switch in debug mode, following error is thrown is instructions are sent for the start table i.e. table ID 0.

In order to pass instructions into any other table other than start table, we change the parameters in OFPFlowMod function of the RYU application. Here by default the table ID parameter is set to 0, which we change to 100 so that flow rules are passed to hardware table of the switch.

Once the error is fixed, a successful connection of the RYU application with the switch will look like this

Step 5: Using IXP Manager as a peering web portal between ISPs

About IXP Manager:

IXP Manager is a full stack management system for Internet eXchange Points (IXPs) which includes an administration and customer portal; provides end to end provisioning; and both teaches and implements best practice. There are about 80 IXPs using IXP Manager around the world. There are lots of customer portal features and administrative/automation features that IXP manager give to us. Take the high level view of what features it give to use from ixp manager site mentioned in resources. Documentation link is also given in resources.

Reason to Use IXP Manager:

Coming to the reason why we used IXP manager is that it provides us a customer/ISP portal where customers/ISPs can request peerings to other member ISPs through emails independent of IXP staff coming between them. Through customer portal, IXP manager provides customers/ISPs the relevant information like AS number for peering with other members.

IXP Manager Support from INEX:

We asked from IXP manager community some support related questions. I am giving the link of our conversations in resources. For the help, we have to subscribe for the ixp manager support. Link is given in resources for subscription.

SDX Architecture Using IXP manager server, PCs and Aruba Switch (JL258A)

IXP manager should be in same subnet and attached to switch as shown as PC4 in the architecture below..

Installation of IXP Manager:

For installation, use the link in resources or follow our steps.

The recommended platform for the v5 branch of IXP Manager is Ubuntu LTS 18.04.

To install on this platform, please proceed as follows:

1. Prepare a physical / virtual machine with (minimum) 8GB of disk space and 2GB of RAM.
2. Attach / insert the latest Ubuntu 18.04 LTS 64-bit PC (AMD64) server install image and boot.
3. At the initial menu where you choose Install Ubuntu Server, first:
   • Press F4
   • If installing on a physical server, select Install a minimum system
   • If installing on a virtual server, select Install a minimal virtual machine
4. Now select Install Ubuntu Server and step through the various options and configure as you like until:
5. When you reach the Software selection screen, select only OpenSSH Server and then complete the installation and reboot.
6. When your new server has rebooted, log in and Now type following:
   # change to root user sudo su - # download the installation script wget https://github.com/inex/IXP-Manager/raw/master/tools/installers/ubuntu-lts-1804-ixp-manager-v5.sh # and execute it: bash ./ubuntu-lts-1804-ixp-manager-v5.sh
7. You will be asked to put information about username and IXP. Then in the end, the congratulation message with IP address of IXP manager and admin username/password. Save this username and password for future purposes. But you can see this message by typing:
   sudo nano ixp-manager-install-details.txt
   The message will look something like the screenshot above.
8. In case, there is some error you face while logging in or you didnt receive the congratulations message. Run the installation script again by running command 3 of point 6.
9. Now, the installation is done and IXP manager will be accessible on IP address of server through web browser of any PC within same subnet. You can also access the IXP manager by accessing localhost in browser after installing GUI in Ubuntu Server 18.04.

Post-install Steps:

Now, our purpose is to make peering manager (which is in customer portal) work so that customers can maintain and contact potential peers. As IXP manager and also the peering manager in it are information portal. We need to fill in the information our infrastructure, facilities, racks and switches so that ISPs can peer.

For that purpose, we followed this link: https://docs.ixpmanager.org/install/next-steps/

1. Login to IXP manager through admin username and password.
2. Create your infrastructures. Generally, an infrastructure represents a collection of switches which form an IXP's peering LAN. The best way to think of an infrastructure is to think of it as an IXP. - One infrastructure was already there. So, there is no need usually to add infrastructure. The detail of infrastructure we added is above in the screenshot.
   
3. Infrastructure is pre-requisite for facilities. Add your facilities. We added one facility with the following detail in picture.
   
4. Add your racks. You will need this later to add patch panels and switches. We added one rack with following details.
   
5. You can now add switches. We added switch with following sample and compulsory details. Give IP address of your switch. SNMP communityś default value is usually public in all switches.
   
6. Add your VLAN(s). We have made one VLAN.
   
7. Add your peering IP addresses. IXP Manager will let you add complete ranges (e.g. /24) and more sensible ranges of IPv6 (i.e. not an entire /64!). We have added IP addresses like this:
   
8. Add customers now. To add route server as a customer, we added the details mentioned below.
   
9. To add ISP1 as a customer, we added the details mentioned below in screenshot. Similarly, we added ISP2 with AS number 7676.
   
10. To add ports of switch, go to switch → Switch ports and press + sign. We generated 8 ports like this and name them afterwards to add them.
    
11. Now, go to interfaces/ports and press + button to add interface wizard. Do this for ISP1, ISP2 and Route server separately. Add Physical interfaces and Vlan interfaces according to your system. Sample setting is given:
    
12. Now from admin portal, you can login as customer. Add a user first for a customer. Add it through the Users tab in IXP customer actions. Now login as that user as shown in the screenshot below:
    
13. Now a customer can go to peering manager and request for peering and maintain its peering list. We can see ISP1 and ISP2 in its peering list. Now, ISP1 can request peering to ISP2 by mailing it.

Resources:

1. “Basic BGP Configuration.” CCNA Training » Basic BGP Configuration, www.9tut.com/basic-bgp-configuration.
2. “How to Build a Network of Linux Routers Using Quagga.” Open-Source Routing and Network Simulation, 25 Oct. 2016, www.brianlinkletter.com/how-to-build-a-network-of-linux-routers-using-quagga/.
3. “Cisco BGP route server configuration”,https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/15-mt/irg-15-mt-book/irg-route-server.pdf
4. “Quagga for BGP Testing”, http://gregsowell.com/?p=543
5. “Switching Hub.” Switching Hub - Ryubook 1.0 Documentation, osrg.github.io/ryu-book/en/html/switching_hub.html?fbclid=IwAR1gfPDkvEo5xQ6xFFeGO0b9hcxx5w_y7x049Ymn-xR11yP958MiaxdDY9A.
6. “Installation of IXP Manager”, https://github.com/inex/IXP-Manager/tree/master/tools/installers
7. “Features of IXP Manager”, https://www.ixpmanager.org/
8. “Documentation” https://docs.ixpmanager.org/
9. “Conversation with INEX or Mailing List of July”, Messages by Haider Ali, https://www.inex.ie/pipermail/ixpmanager/2019-July/
10. “Conversation with INEX or Mailing List of August”, Messages by Haider Ali, https://www.inex.ie/pipermail/ixpmanager/2019-August/
11. “Subscription for INEX mailing list”, https://www.inex.ie/mailman/listinfo/ixpmanager

Project Evaluation

It has been worth the time spent in addressing challenges during this project as it has the impact not only in developing countries but also worldwide as we intend to make an open source contribution in IXP manager (in addition to the original proposal). Also, as a future goal, we aim to do measurement studies on testbed of IXP in Islamabad after full-scale deployment of our product, which will solve many challenges faced by ISPs today and attract CDNs or companies like facebook towards developing countries like Pakistan. Innovations like an open source contribution and publication in top conferences stand to be some of the desirable outcomes of this project.

This project has the potential of helping developing countries like Pakistan to create new IXPs in Lahore and Karachi. This will also save the operating costs and ease the IXP’s operations. All these benefits and strengths of project add value to the organization in terms of the significant role played by the project team for benefitting not only the developing countries but also the global IXPs who intend to SDN-dnable their infrastructure.

The project has achieved all of its major objectives of setting up the Software defined Internet Exchange Point and creating manuals for everyone to view and benefit. Additional objectives will lead to continuously evolving activities such as implementation of additional peering applications (e.g., application-specific peering), new training programs and measurement studies in the wild.

This project has helped individuals involved in the project to learn the practicalities when implementing and/or configuring the SDN, BGP and IXP. Challenges came up often and solving those played significant role in the learning and professional growth of the team.

There are few reasons that this project is a success. A main reason is that the team was able to acquire real openflow switches (thanks to the project funding) and by using those switches, we were able to automate peering and applications like application-specific peering in real environments and perform measurement studies on real SDN switches. The design decision of using OVS first helped the project team become an expert in BGP setup and configuration. Overall, support of organizations like ISIF Asia and LUMS, and expertise of the research team has helped a lot towards success of the project and will continue to serve as a strong stepping stone for pursuing further directions stemming from this project.

Indicators	Baseline	Project activities related to indicator	Outputs and outcomes	Status
How do you measure project progress, linked to the your objectives and the information reported on the Implementation and Dissemination sections of this report.	Refers to the initial situation when the projects haven’t started yet, and the results and effects are not visible over the beneficiary population.	Refer to how the project has been advancing in achieving the indicator at the moment the report is presented. Please include dates.	We understand change is part of implementing a project. It is very important to document the decision making process behind changes that affect project implementation in relation with the proposal that was originally approved.	Indicate the dates when the activity was started. Is the activity ongoing or has been completed? If it has been completed add the completion dates.
Implementation of IXP using Route Server 2 ISPs (2 PCs) and Open Virtual Switch (OVS)	We wanted to make the prototype by using the baselines provided by quagga and OVS	Configurations of Quagga for 2 ISPs in 2 PCs, Configurations for route server in 3rd PC and Configurations of OVS in 3rd PC are the activities related to indicator as mentioned in manual.	Communication between these four components (2 ISPs, route server and OVS) was the outcome giving us the working prototype of IXP.	Status: completed, Start Date: November, 2018, Completion date: 28th February, 2019
Implementation of Ryu Controller and Open Virtual Switch (OVS)	IXP prototype developed earlier and the open source Ryu controller serve as the baseline.	Following are the activities as explained in detail in manual: 1- Passing flow rules to switch using RYU controller 2- RYU Installation on IXP(PC3) 3- Attaching Controller with Switch(OVS)	Connection between Ryu Controller and Open Virtual Switch as explained in manual.	Status: completed, Start Date: 1st March, 2019, Completion date: 15th June, 2019
Configuration of Aruba Open flow Switch and replacing OVS with Aruba switch	Buying of Aruba Switch and development of Software defined exchange point based on OVS serve as the baselines.	Following are the activities as explained in manual in detail: 1- SDX Architecture Using PCs and Aruba Switch (JL258A) 2- Setup Details of Aruba Switch 3-Switch Configuration 4-Ryu Controller connection with Aruba Switch	Aruba Switch was accurately configured and connected to Ryu controller as the outcome.	Status: completed, Start Date: 15th June, 2019, Completion date: 15th July, 2019
IXP Manager installation, configurations and connection with switch	The open source documentation of IXP Manager serve as the baseline of this indicator.	Following are the activities as mentioned in manual in detail: 1- IXP Manager as a peering web portal between ISPs 2- Installation of IXP Manager: 3- Post-install Steps and configurations of IXP manager	IXP Manager installation, configurations and connection with switch successfully achieved.	Status: completed, Start Date: 15th July, 2019, Completion date: 20th August, 2019
Implementation of applications like application-specific peering, inbound traffic engineering etc.	OpenFlow framework serves as a baseline for the achievement of this indicator.	It is currently being implemented by writing scripts in Openflow. Outcomes are yet to come.	Outcomes are yet to come.	Status: ongoing, Start Date: 15th August, 2019
Connection of IXP Manager with the Controller	The IXP Manager and Ryu Controller implemented previously serve as a baseline.	All activities are currently being implemented: 1- The development of the controller’s user interface in IXP Manager 2- API development for the connection between IXP Manager and Ryu Controller	Connection of IXP Manager with the Controller	Status: ongoing, Start Date: 15th August, 2019
Measurement studies on real testbed at Islamabad IXP or on real switches using datasets available for public use	Baselines like implementation of novel SDN applications are being implemented.	Activities will be planned once baselines will be achieved.	Not yet started.	Status: not yet started

Gender Equality and Inclusion

The project team itself could not get direct contributions from a female though the university not just provides equal opportunity without a gender bias but does encourage and facilitate females to take part in scientific studies. This lack of direct female representation on the project team can perhaps be attributed to the global trend of a samller proportion of females in the computing area, and an even smaller in the networking systems area, particularly the one which invovles working with networking hardware, as was the case in this project. The team, howver, made sure that female participation is encouraged in all the information series lectures and seminars conducted during the course of this project, in line with teh goals of the university.

Project Communication Strategy

Pakistan is the sixth largest country in the world with over 200M population and an internet penetration approx 20%. The country is served by a few tens of internet service providers, small and large, nation-wide and regional. Unfortunately, the IXP initiative is relatively new in Pakistan with first IXP established only recently (in 2017). The lead researcher of this project serves on the IXP board and the project team played a key role in making the IXP possible. Despite this, the inter-provider connectivity structure in the country is far from desirable. This project will not only provide this platform to easily interconnect and manage ISPs but will also pave ways for global content and application service providers and CDNs to peer at the IXP. This will be possible by having easy access to the inter-carrier traffic volumes observed at the IXP, easily measureable by the use of SDN.

The project team is already engaged with the IXP operations and this project will only strengthen the relationship between the academic community with the service provider community. Another outcome we expect from the project is the training of human resource in understand inter-ISP connectivity at a large scale. This will help the general service provider body to run their operations more efficiently and without running into routing issues stemming from lack of operational expertise (case in point, the inadvertent blockage of YouTube for most of the world for a couple of hours, back in 2008 due to a BGP misconfiguration error by a large service provider from Pakistan).

Recommendations and Use of Findings

Following are the recommendations to researchers conducting similar projects:

• Use the same setup as we used, in this way, we will be able to provide support or training programs. Setup includes ryu controller, Aruba Openflow switch, quagga for ISPs (autonomous systems), BGP and route server configurations, IXP manager as a web portal, SDN applications like application-specific peering and inbound traffic engineering etc.
• If you don’t use the same setup, still it’s not difficult to implement SDN based IXP if manuals and documentations of setup components (components here means controller, switch etc) are available. There are many types of controllers and switches available with documentations. We found ryu controller and Aruba Switch feasible as we have worked with both of these components in previous projects.
• There are other web based portals similar to IXP manager exist too. One of the portals worth mentioning here for recommendation is the Toulouse SDN based Internet exchange point manager. It is important because they have created it specially for SDN based IXP and is also openly available for other IXP’s use. There were few problems with Toulouse SDX manager that it was created recently and will not have much impact as it is nascent and not yet fully developed. There were more benefits of using IXP manager and developing package for enabling SDN in IXP manager than using Toulouse SDX manager. Firstly, IXP manager is being used by 79 IXPs. Secondly, it has been continuously improved by INEX since it was created in 2007. IXP manager has proven itself beneficial with many features such as peering manager, looking glass, security etc. If we convert this IXP manager into SDN enabled IXP manager then it will not only help developing countries like Pakistan but also it will help 79 IXPs to deploy SDN enabled IXP manager with minimal effort of installing our package. IXP manager also provides automation of switches like Cumulus and Arista. If one wants to automate every component through the web portal and also, if one has the option to use above mentioned switches then INEX has automation support available.  But as far as recommendation is concerned, if someone wants to use web portal then one can use IXP manager or Toulouse SDX manager according to one’s needs.
• We found the applications like application-specific peering and inbound traffic engineering easy and feasible to implement according to our setup as compared to applications like server load balancing and traffic redirection through middleboxes etc. But we will implement these latter mentioned applications after implementing former mentioned applications. Our recommendations to research groups are to go with the same order as we went and seek help in implementing above mentioned applications from us, if they face problems or from SDX community. Also, we advise them to find new compelling applications in SDX, which is also our goal.
• One of the topics which need to be worked on is the scalability analysis of SDN based IXP on real IXP, which is also our goal after implementing SDX on our current setup. We recommend research groups to work on this topic of scalability of SDX.

Bibliography

1.  “Basic BGP Configuration.” CCNA Training » Basic BGP Configuration, www.9tut.com/basic-bgp-configuration.
2. “How to Build a Network of Linux Routers Using Quagga.” Open-Source Routing and Network Simulation, 25 Oct. 2016, www.brianlinkletter.com/how-to-build-a-network-of-linux-routers-using-quagga/.
3. “Cisco BGP route server configuration”,https://www.cisco.com/c/en/us/td/docs/ios-xml/ios/iproute_bgp/configuration/15-mt/irg-15-mt-book/irg-route-server.pdf
4. “Quagga for BGP Testing”, http://gregsowell.com/?p=543
5. “Switching Hub.” Switching Hub - Ryubook 1.0 Documentation,
   osrg.github.io/ryu-book/en/html/switching_hub.html?fbclid=IwAR1gfPDkvEo5xQ6xFFeGO0b9hcxx5w_y7x049Ymn-xR11yP958MiaxdDY9A.
6. “Installation of IXP Manager”, https://github.com/inex/IXP-Manager/tree/master/tools/installers
7. “Features of IXP Manager”, https://www.ixpmanager.org/
8. “Documentation” https://docs.ixpmanager.org/
9. “Conversation with INEX or Mailing List of July”, Messages by Haider Ali, https://www.inex.ie/pipermail/ixpmanager/2019-July/
10. “Conversation with INEX or Mailing List of August”, Messages by Haider Ali, https://www.inex.ie/pipermail/ixpmanager/2019-August/
11. “Subscription for INEX mailing list”, https://www.inex.ie/mailman/listinfo/ixpmanager

 

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License