We have all heard, at one time or another, a system administrator blaming the network. Then, it is up to the network engineers to prove that the network is not in cause. To do this, a minimum of network troubleshooting skills on Linux systems could be required.
For the first post in the series: many examples of how to use the ip and nmcli commands.
Other posts of the series
This post is part of a series of Linux Networking tips and tricks.
The other posts of this series are:
The ip commands: IP queries (the “show” commands)
Let’s start with basic ip queries checks, or “show commands” in our Cisco world.
Link status
The ip link show command will show information for all interfaces:
$ ip link show 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2: enp0s31f6: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN mode DEFAULT group default qlen 1000 link/ether e8:6a:64:5d:1e:66 brd ff:ff:ff:ff:ff:ff 3: wlp61s0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state DOWN mode DORMANT group default qlen 1000 link/ether 98:3b:8f:11:3a:0f brd ff:ff:ff:ff:ff:ff 4: enp0s20f0u4u1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000 link/ether 3c:e1:a1:46:ae:e5 brd ff:ff:ff:ff:ff:ff
In this example, we can see 4 interfaces:
- lo: is the loopback interface
- enp0s31f3: is an ethernet interface. We can see the state is “DOWN”.
- wlp61s0: is a wireless interface. The state is “DOWN” too.
- enp0s20f0u4u1: is another ethernet interface. Here the state us “UP”.
A more human-readable output is with the -brief option, which can be shortened to -br :
(Right, like “show ip int brief”)
$ ip -br link show lo UNKNOWN 00:00:00:00:00:00 <LOOPBACK,UP,LOWER_UP> enp0s31f6 DOWN e8:6a:64:5d:1e:66 <NO-CARRIER,BROADCAST,MULTICAST,UP> wlp61s0 DOWN 0e:6a:82:81:5e:b5 <NO-CARRIER,BROADCAST,MULTICAST,UP> enp0s20f0u4u1 UP 3c:e1:a1:46:ae:e5 <BROADCAST,MULTICAST,UP,LOWER_UP>
You can add colors with the -c[olor] option, to see DOWN in red and UP in green:
$ ip -br -c link show lo UNKNOWN 00:00:00:00:00:00 <LOOPBACK,UP,LOWER_UP> enp0s31f6 DOWN e8:6a:64:5d:1e:66 <NO-CARRIER,BROADCAST,MULTICAST,UP> wlp61s0 DOWN 0e:6a:82:81:5e:b5 <NO-CARRIER,BROADCAST,MULTICAST,UP> enp0s20f0u4u1 UP 3c:e1:a1:46:ae:e5 <BROADCAST,MULTICAST,UP,LOWER_UP>
Of course, in the case of many interfaces, you can specify to see only one, like this:
$ ip -br link show enp0s20f0u4u1 enp0s20f0u4u1 UP 3c:e1:a1:46:ae:e5 <BROADCAST,MULTICAST,UP,LOWER_UP>
Interfaces statistics
Now, to see interfaces statistics, we can use the -s option:
$ ip -s link 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000 link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 RX: bytes packets errors dropped overrun mcast 0 0 0 0 0 0 TX: bytes packets errors dropped carrier collsns 0 0 0 0 0 0 2: enp0s31f6: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc fq_codel state DOWN mode DEFAULT group default qlen 1000 link/ether e8:6a:64:5d:1e:66 brd ff:ff:ff:ff:ff:ff RX: bytes packets errors dropped overrun mcast 0 0 0 0 0 0 TX: bytes packets errors dropped carrier collsns 0 0 0 0 0 0 3: wlp61s0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc noqueue state DOWN mode DORMANT group default qlen 1000 link/ether 86:0c:27:8b:c2:7c brd ff:ff:ff:ff:ff:ff RX: bytes packets errors dropped overrun mcast 44535911 42246 0 89 0 0 TX: bytes packets errors dropped carrier collsns 4857683 21037 0 0 0 0 4: enp0s20f0u4u1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000 link/ether 3c:e1:a1:46:ae:e5 brd ff:ff:ff:ff:ff:ff RX: bytes packets errors dropped overrun mcast 30723925 39013 0 96 0 0 TX: bytes packets errors dropped carrier collsns 4132292 23590 0 0 0 0
For one specific interface, same as above:
$ ip -s link show enp0s20f0u4u1 4: enp0s20f0u4u1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP mode DEFAULT group default qlen 1000 link/ether 3c:e1:a1:46:ae:e5 brd ff:ff:ff:ff:ff:ff RX: bytes packets errors dropped overrun mcast 33023147 41455 0 100 0 0 TX: bytes packets errors dropped carrier collsns 4416126 25037 0 0 0 0
Display the MAC address table
To see the content of the MAC address table, we use the ip neigh command:
$ ip neigh 10.0.2.2 dev enp0s3 lladdr 52:54:00:12:35:02 STALE
Display the IP information
Now, we would like to check the IP address we have. I use the -br option here too for a more readable output:
Please note: I use a different machine than above for the next examples
$ ip -br addr lo UNKNOWN 127.0.0.1/8 ::1/128 enp0s3 UP 10.0.2.15/24 fe80::2dd2:9ab:5892:352d/64
In the case of many interfaces, we can show only one specific interface:
$ ip -br address show dev enp0s3 enp0s3 UP 10.0.2.15/24 fe80::2dd2:9ab:5892:352d/64
Display the routing table
Use the command ip route to see the routing table
$ ip route default via 10.0.2.2 dev enp0s3 proto dhcp metric 100 10.0.2.0/24 dev enp0s3 proto kernel scope link src 10.0.2.15 metric 100
This is a very simple topology, so I only have a default gateway (10.0.2.2) here.
Display the route for an address
In the case of many routes with different metrics, we can display the route an address will use, with the command: ip route get
$ ip route get 8.8.8.8 8.8.8.8 via 10.0.2.2 dev enp0s3 src 10.0.2.15 uid 1000 cache
Here, we can see for destination 8.8.8.8 the next hop will be 10.0.2.2 (the default gateway on the previous example), via the local interface: enp0s3.
Making changes
Now, let’s see some examples of how to make changes to our network configuration.
Important note: before diving into the examples, it should be noted that there are many differences on how the networking part is managed on RHEL/CentOS, between the previous and current version. This post is based on CentOS 8.0 using NetworkManager (the default).
For example, by default with RHEL/CentOS 8, if you try to use a legacy network.service unit file, it is not available and you will get an error:
# systemctl restart network Failed to restart network.service: Unit network.service not found.
Bring up/down an interface
With ip commands
- To bring down our enp0s3 interface:
# ip link set enp0s3 down
- Now, let’s see the status:
# ip -br addr lo UNKNOWN 127.0.0.1/8 ::1/128 enp0s3 DOWN 10.0.2.20/24
- Bring it back up and check the status:
# ip link set enp0s3 up # ip -br addr lo UNKNOWN 127.0.0.1/8 ::1/128 enp0s3 UP 10.0.2.20/24
With nmcli commands
- To enable the interface:
# nmcli device connect enp0s3
- And to disable it:
# nmcli device disconnect enp0s3
Change the IP address
Here are some examples where we want to change our IP address, from a DHCP IP to a static IP address.
The initial state is: we have one interface named enp0s3 configured in DHCP. And our goal is to set the static IPv4 address 10.0.2.200/24.
With the nmcli commands
As RHEL/CentOS 8 uses NetworkManager, we can use the nmcli commands to make changes.
- We assign the IPv4 10.0.2.200 to enp0s3 interface:
# nmcli con mod enp0s3 ipv4.addresses 10.0.2.200/24
- Now, set the gateway to 10.0.2.2:
# nmcli con mod enp0s3 ipv4.gateway 10.0.2.2
- Change the configuration from dhcp to static:
# nmcli con mod enp0s3 ipv4.method manual
- Set the DNS value to 1.1.1.1:
# nmcli con mod enp0s3 ipv4.dns "1.1.1.1"
- We can now save the changes and reload the interface:
# nmcli con up enp0s3 Connection successfully activated (D-Bus active path: /org/freedesktop/NetworkManager/ActiveConnection/4)
- With the ip -br a command, we can check the change is done:
# ip -br a lo UNKNOWN 127.0.0.1/8 ::1/128 enp0s3 UP 10.0.2.200/24 fe80::2dd2:9ab:5892:352d/64
- We can also see nmcli updated the network-script file:
[root@centest ~]# cat /etc/sysconfig/network-scripts/ifcfg-enp0s3 TYPE=Ethernet PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=none DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes IPV6_FAILURE_FATAL=no IPV6_ADDR_GEN_MODE=stable-privacy NAME=enp0s3 UUID=6ae9864a-ea58-4e53-a2bb-54c9abfebe03 DEVICE=enp0s3 ONBOOT=yes IPADDR=10.0.2.200 PREFIX=24 GATEWAY=10.0.2.2 DNS1=1.1.1.1
With network-script
Now, let’s revert our change using the network-script.
- I edited the file /etc/sysconfig/network-script/ifcfg-enp0s3 this way:
# vi /etc/sysconfig/network-scripts/ifcfg-enp0s3 (...edit the file...) # cat /etc/sysconfig/network-scripts/ifcfg-enp0s3 TYPE=Ethernet PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=dhcp DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes IPV6_FAILURE_FATAL=no IPV6_ADDR_GEN_MODE=stable-privacy NAME=enp0s3 UUID=6ae9864a-ea58-4e53-a2bb-54c9abfebe03 DEVICE=enp0s3 ONBOOT=yes
- After a restart of NetworkManager, we have the dhcp ip again:
# systemctl restart NetworkManager.service # ip -br a lo UNKNOWN 127.0.0.1/8 ::1/128 enp0s3 UP 10.0.2.15/24 fe80::2dd2:9ab:5892:352d/64
With nmtui
Another method is to use the NMTUI tool.
- Type: nmtui ou nmtui edit interfacename to start this tool.
- Then, it is a very straightforward graphical interface, no need for special instructions here:
Create 802.1q sub-interfaces and add an IP to it
Now let’s make a little bit more advanced networking.
Let’s take an example where we have one physical ethernet interface, and we want to use it as a trunk with different sub-interfaces.
With nmcli commands
- Let’s create two sub-interfaces, with VLAN tagging 10 and 20 and IP 192.168.10.200/24 and 192.168.20.200/24:
# nmcli con add type vlan con-name vlan-enp0s3.10 ifname enp0s3.10 dev enp0s3 id 10 ip4 192.168.10.200/24 Connection « vlan-enp0s3.10 » (57d81e20-7777-44e9-92c2-bd3a8656f189) successfully added. # nmcli con add type vlan con-name vlan-enp0s3.20 ifname enp0s3.20 dev enp0s3 id 20 ip4 192.168.20.200/24 Connection « vlan-enp0s3.20 » (786deb1f-d8de-491f-b4b3-ff3e504e4af9) successfully added.
- We can see the interfaces are created, with the nmcli or with the ip command:
# nmcli connection NAME UUID TYPE DEVICE enp0s3 6ae9864a-ea58-4e53-a2bb-54c9abfebe03 ethernet enp0s3 vlan-enp0s3.10 57d81e20-7777-44e9-92c2-bd3a8656f189 vlan enp0s3.10 vlan-enp0s3.20 786deb1f-d8de-491f-b4b3-ff3e504e4af9 vlan enp0s3.20 # ip -br link show lo UNKNOWN 00:00:00:00:00:00 <LOOPBACK,UP,LOWER_UP> enp0s3 UP 08:00:27:f0:95:7f <BROADCAST,MULTICAST,UP,LOWER_UP> enp0s3.10@enp0s3 UP 08:00:27:f0:95:7f <BROADCAST,MULTICAST,UP,LOWER_UP> enp0s3.20@enp0s3 UP 08:00:27:f0:95:7f <BROADCAST,MULTICAST,UP,LOWER_UP> # ip -br a lo UNKNOWN 127.0.0.1/8 ::1/128 enp0s3 UP 10.0.2.15/24 fe80::2dd2:9ab:5892:352d/64 enp0s3.10@enp0s3 UP 192.168.10.200/24 fe80::619:8794:fc06:5429/64 enp0s3.20@enp0s3 UP 192.168.20.200/24 fe80::7084:5d10:d339:4d72/64
- We can see the interfaces are created into /etc/sysconfig/network-script directory. And let’s see the content of one interface:
# ls -la /etc/sysconfig/network-scripts/ total 16 drwxr-xr-x. 2 root root 82 18 oct 12:38 . drwxr-xr-x. 7 root root 4096 16 oct 14:44 .. -rw-r--r--. 1 root root 267 18 oct 12:36 ifcfg-enp0s3 -rw-r--r--. 1 root root 388 18 oct 12:38 ifcfg-vlan-enp0s3.10 -rw-r--r--. 1 root root 388 18 oct 12:38 ifcfg-vlan-enp0s3.20 # cat /etc/sysconfig/network-scripts/ifcfg-vlan-enp0s3.10 VLAN=yes TYPE=Vlan PHYSDEV=enp0s3 VLAN_ID=10 REORDER_HDR=yes GVRP=no MVRP=no PROXY_METHOD=none BROWSER_ONLY=no BOOTPROTO=none IPADDR=192.168.10.200 PREFIX=24 DEFROUTE=yes IPV4_FAILURE_FATAL=no IPV6INIT=yes IPV6_AUTOCONF=yes IPV6_DEFROUTE=yes IPV6_FAILURE_FATAL=no IPV6_ADDR_GEN_MODE=stable-privacy NAME=vlan-enp0s3.10 UUID=57d81e20-7777-44e9-92c2-bd3a8656f189 DEVICE=enp0s3.10 ONBOOT=yes
- As you can see below, the interfaces are immediately up and we can ping them:
# ping 192.168.10.200 PING 192.168.10.200 (192.168.10.200) 56(84) bytes of data. 64 bytes from 192.168.10.200: icmp_seq=1 ttl=64 time=0.284 ms 64 bytes from 192.168.10.200: icmp_seq=2 ttl=64 time=0.090 ms 64 bytes from 192.168.10.200: icmp_seq=3 ttl=64 time=0.253 ms ^C --- 192.168.10.200 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 76ms rtt min/avg/max/mdev = 0.090/0.209/0.284/0.085 ms # ping 192.168.20.200 PING 192.168.20.200 (192.168.20.200) 56(84) bytes of data. 64 bytes from 192.168.20.200: icmp_seq=1 ttl=64 time=0.153 ms 64 bytes from 192.168.20.200: icmp_seq=2 ttl=64 time=0.172 ms 64 bytes from 192.168.20.200: icmp_seq=3 ttl=64 time=0.131 ms ^C --- 192.168.20.200 ping statistics --- 3 packets transmitted, 3 received, 0% packet loss, time 42ms rtt min/avg/max/mdev = 0.131/0.152/0.172/0.016 ms
Add / Delete / Replace an entry to the routing table
Now, let’s take a look at my routing table:
# ip route 192.168.10.0/24 dev enp0s3.10 proto kernel scope link src 192.168.10.200 metric 400 192.168.20.0/24 dev enp0s3.20 proto kernel scope link src 192.168.20.200 metric 401
As you can see, my routing table is composed of the two connected routes: 192.168.10.0/24 and 192.168.20.0/24 corresponding to the two sub-interfaces created above. But, I have no default-route anymore.
With ip commands:
- Let’s create one via 192.168.10.1 on enp0s3.10 and check the result:
# ip route add default via 192.168.10.1 dev enp0s3.10
# ip route
default via 192.168.10.1 dev enp0s3.10
192.168.10.0/24 dev enp0s3.10 proto kernel scope link src 192.168.10.200 metric 400
192.168.20.0/24 dev enp0s3.20 proto kernel scope link src 192.168.20.200 metric 401
# ip route get 8.8.8.8
8.8.8.8 via 192.168.10.1 dev enp0s3.10 src 192.168.10.200 uid 0
cache
- To add a static route:
# ip route add 10.0.0.0/16 via 192.168.20.1 dev enp0s3.20 # ip route add 11.0.0.0/16 via 192.168.20.1 dev enp0s3.20 # ip route default via 192.168.10.1 dev enp0s3.10 10.0.0.0/16 via 192.168.20.1 dev enp0s3.20 11.0.0.0/16 via 192.168.20.1 dev enp0s3.20 192.168.10.0/24 dev enp0s3.10 proto kernel scope link src 192.168.10.200 metric 400 192.168.20.0/24 dev enp0s3.20 proto kernel scope link src 192.168.20.200 metric 401
- To delete a route:
# ip route delete 11.0.0.0/16 via 192.168.20.1 dev enp0s3.20 # ip route default via 192.168.10.1 dev enp0s3.10 10.0.0.0/16 via 192.168.20.1 dev enp0s3.20 192.168.10.0/24 dev enp0s3.10 proto kernel scope link src 192.168.10.200 metric 400 192.168.20.0/24 dev enp0s3.20 proto kernel scope link src 192.168.20.200 metric 401
Note: these changes are lost after a reboot of the machine.
With nmcli commands
- First, list the devices:
# nmcli conn NAME UUID TYPE DEVICE enp0s3 6ae9864a-ea58-4e53-a2bb-54c9abfebe03 ethernet enp0s3 vlan-enp0s3.10 57d81e20-7777-44e9-92c2-bd3a8656f189 vlan enp0s3.10 vlan-enp0s3.20 786deb1f-d8de-491f-b4b3-ff3e504e4af9 vlan enp0s3.20
- Then, add a new static route. For example, let’s add 192.168.222.0/24 to 192.168.10.1:
# nmcli con modify vlan-enp0s3.10 +ipv4.routes "192.168.222.0/24 192.168.10.1"
- I reset the interface and check the routing table:
# nmcli con up vlan-enp0s3.10 # ip route 192.168.10.0/24 dev enp0s3.10 proto kernel scope link src 192.168.10.200 metric 401 192.168.20.0/24 dev enp0s3.20 proto kernel scope link src 192.168.20.200 metric 400 192.168.222.0/24 via 192.168.10.1 dev enp0s3.10 proto static metric 401
- How to add a default gateway on the interface VLAN20:
# nmcli con mod vlan-enp0s3.20 ipv4.gateway "192.168.20.1" # nmcli con up vlan-enp0s3.20 # ip route default via 192.168.20.1 dev enp0s3.20 proto static metric 402 192.168.10.0/24 dev enp0s3.10 proto kernel scope link src 192.168.10.200 metric 401 192.168.20.0/24 dev enp0s3.20 proto kernel scope link src 192.168.20.200 metric 402 192.168.222.0/24 via 192.168.10.1 dev enp0s3.10 proto static metric 401
Note: these changes are persistent after a reboot of the machine.
- To add more arguments, like a metric or anything else, the command is:
# nmcli connection modify connection_name ipv4.routes "ip[/prefix] [next_hop] [metric] [attribute=value] [attribute=value] ..."
IP Forwarding
Forwarding (routing) between the interfaces of a Linux machine is disabled by default for security reasons.
To enable it, we have to change the default value of 0 in file /proc/sys/net/ipv4/ip_forward to 1
This can be done by editing the file (vi or another editor) or by using the command:
#echo 1 > /proc/sys/net/ipv4/ip_forward
Read more
ip Command Cheat Sheet for Red Hat Enterprise Linux
A beginner’s guide to network troubleshooting in Linux
RHEL 8 – CONFIGURING AND MANAGING NETWORKING
Other posts of the series
This post is part of a series of Linux Networking tips and tricks.
The other posts of this series are:
For Centos 7+
service network restart
should still work. Beware of doing it remotely – it the reconfiguration involves changing IP on main interface you will get disconnected so it’s better to to via atd thus
at now + 1 minute
service network restart
^d