Anyone Setup an AWS Direct Connect or Azure ExpressRoute on a Catalyst Running IOS XE yet?

[chrisautomates]

Looking for confirmation that it is possible and any particular quirks / settings you are able to share.

[alemabrahao]

Azure has a document showing how to configure it; AWS doesn't have a specific document about IOS-XE, but it probably follows the same pattern, so I believe it's possible as well.

Remember that we're talking about IOS-XE without Meraki management, because I believe Meraki doesn't allow for all configurations.

Azure ExpressRoute: Router configuration samples | Microsoft Learn

[chrisautomates]

Oh I should have specified, I meant IOS XE with Cloud Management.

I will take a look.

[alemabrahao]

If you use IOS-XE managed by Cloud and use the Cloud CLI, it might be possible, but I personally haven't tried it myself.

Cloud CLI for Cloud-Managed IOS XE Switches - Cisco Meraki Documentation

[chrisautomates]

Didnt even think about that! 

Guess CLI is the backup in case we do need additional settings adjusted.

[MKutka]

Cloud CLI only support read/write on Config Source: Device mode. On Config Source: Cloud mode CLI is only read therefore you would not be able to configure some of the more nuanced config in the Azure ExpressRoute documentation that was linked above. It appears to leverage QinQ which is today not supported in full Cloud Managed mode.

[chrisautomates]

This article seems to contradict that

https://documentation.meraki.com/Switching/Cloud_Management_with_IOS_XE/Operate_and_Maintain/Cloud_C...

"Cloud CLI for Cloud-Managed IOS XE Switches"

"Cloud CLI terminal configuration mode provides administrators with IOS XE Privileged Exec view, with the ability to run most IOS XE configuration commands."

I thought Cloud Managed was Config Source: Cloud, while Cloud Monitored was Config Source: Device.

Let me know! I still havent upgraded firmwares so I cant verify.

[MKutka]

The Cloud Monitoring terminology has been deprecated. Everything is considered Cloud Managed now, but the key is where the Configuration resides which is either in the Cloud or on-device. 

The reason for this change is because technically Cloud Monitoring was more than just monitoring. We provided the ability to bounce ports, take pcaps, upgrade firmware, etc. That's much more than just monitoring as were making changes to the device via cloud.

Please refer to this documentation and Table 1 which gives a good breakdown of the different capabilities of each mode!

https://documentation.meraki.com/Switching/Cloud_Management_with_IOS_XE

[chrisautomates]

Understood, thank you!

[chrisautomates]

Here is the AWS sample config for an IOS router, any insight as far as features supported is greatly appreciated!

BGP config looks fine but it also uses BFD, though not clear if its optional.

! Amazon Web Services
!=======================================IPV4=======================================
! Direct Connect
! Virtual Interface ID: XXXXXXXX
!
! --------------------------------------------------------------------------------
! Interface Configuration

! interface port-channel 1 (In case of a LAG connection)

interface GigabitEthernet0/1
! channel-group 1 mode passive (In case of a LAG connection)
  no ip address

interface GigabitEthernet0/1.XXXX
! interface port-channel 1.XXXX (In case of a LAG connection)
  description "Direct Connect to your Amazon VPC or AWS Cloud"
  encapsulation dot1Q XXXX
  ip address XXX.XXX.XXX.XXX 255.255.255.248

! Note: Please ensure to remove any individual member port configurations before the EtherChannel is configured.

! --------------------------------------------------------------------------------
! Border Gateway Protocol (BGP) Configuration
!
! BGP is used to exchange prefixes between the Direct Connect Router and your
! Customer Gateway.
!
! If this is a Private Virtual Interface, your Customer Gateway may announce a default route (0.0.0.0/0),
! which can be done with the 'network' and 'default-originate' statements. To advertise other/additional prefixes,
! copy the 'network' statement and identify the prefix you wish to advertise. Make sure the prefix is present in the routing
! table of the device with a valid next-hop.
!
! For Public Virtual Interface, you must advertise public IP prefixes that you own.
!
! The local BGP Autonomous System Number (ASN) (XXXXX) is configured as
! part of your Customer Gateway. If the ASN must be changed, the Customer Gateway
! and Direct Connect Virtual Interface will need to be recreated with AWS.

router bgp XXXXX
  address-family ipv4
   neighbor XXX.XXX.XXX.XXX remote-as XXXXX
   neighbor XXX.XXX.XXX.XXX password XXXXXXXX
   network 0.0.0.0
exit

! --------------------------------------------------------------------------------
! Bidirectional Forwarding Detection (BFD) Configuration
!
! Bidirectional Forwarding Detection (BFD) ensures fast forwarding-path failure detection times for BGP.
! Also provides fast failover to redundant Direct Connect connections.
! An example is provided below:
!

interface GigabitEthernet0/1.XXXX
 bfd interval 300 min_rx 300 multiplier 3
router bgp XXXXX
 neighbor XXX.XXX.XXX.XXX fall-over bfd
! --------------------------------------------------------------------------------
! Local Preference BGP Communities (Optional)
!
! You can use local preference BGP community tags to achieve load balancing and route preference for incoming traffic to your network.
! For each prefix that you advertise over a BGP session, you can apply a community tag to indicate the priority of the associated path for returning traffic.
! The following local preference BGP community tags are supported:
!
!    7224:7100-Low preference
!    7224:7200-Medium preference
!    7224:7300-High preference
!
! Please add the appropriate local preference community tag when advertising prefixes to Amazon using the following example:
!
! ip bgp-community new-format
! ip prefix-list TAG-TO-AWS permit 0.0.0.0/0 le 32
! route-map TO-AWS permit 10
!  match ip address prefix-list TAG-TO-AWS
!  set community XXX.XXX.XXX.XXX
! router bgp XXXXX
!  address-family ipv4
!   neighbor XXX.XXX.XXX.XXX send-community
!   neighbor XXX.XXX.XXX.XXX route-map TO-AWS out

! Additional Notes and Questions
!  - Amazon Web Services Direct Connect Getting Started Guide:
!       http://docs.amazonwebservices.com/DirectConnect/latest/GettingStartedGuide/Welcome.html

        

[MilesMeraki]

Why i've not done this is theoretically is possible, AWS Direct Connects use VLANs and Express Routes use Q-in-Q which appears to be supported on the Catalyst switches/IOS-XE.

Realistically the AWS Direct Connect/Express route is just a layer 2 pseudowire link connection that you would usually terminate onto a router/firewall.

I see most customers terminate these onto their DC Firewall clusters with Zones/Firewall policies applied to limit traffic going over and from the AWS/Azure environments.

Some ISPs can just terminate the Direct Connect/Express Routes into their MPLS WANS VRFs directly and allow the Direct Connect/Express route to terminate directly into the customers WAN VRFS.