System
System Logging
SoodarOS uses systemd-journald as the central logging solution.
- debug service snmp
Enable logging for SNMP service. All SNMP logs appear in journald.
- debug service mender
Enable logging for mender update service. All mender logs appear in journald.
- debug service ntpd
Enable logging for NTP service. All NTP logs appear in journald.
- debug service dhcp4
Enable logging for DHCP4 server service. All logs appear in journald.
- debug dplane fib
Enable data plane( VPP) FIB logs.
- debug dplane ipsec
Enable data plane( VPP) IPSec logs.
- log rotate max-file-size SIZE
set
SIZEas the limit of how sizeable individual journal files may grow at most. When a limit is reached, it rotates to the next journal file.
- log rotate max-files (1-1000)
control how many individual journal files to keep at most. Default is 100.
- log rotate max-use <SIZE|PERCENTAGE>
Control how much disk space the journal may use up at most. The
SIZEis capped at 4G. After reaching the limit, it starts removing elder journal files.
- log rotate max-file-life (1-1000)
The maximum time( in days) to store entries in a single journal file before rotating to the next one.
- log rotate max-retention (1-1000)
The maximum time( in days) to store journal entries. This controls whether journal files containing entries older than the specified period are deleted.
- log file [LEVEL]
If you want to enable log into a file, please enter the command as in this example:
log file informational
If the optional second argument specifying the logging level is not present, the default logging level (typically debugging, but can be changed using the deprecated
log trapcommand) is used. Thenoform of the command disables logging to a file.
- log syslog [LEVEL]
Enable logging output to syslog. If the optional second argument specifying the logging level is not present, the default logging level (typically debugging, but can be changed using the deprecated
log trapcommand) is used. Thenoform of the command disables logging to syslog. Default log level for syslog is set toerrorlevel.
- log syslog [A.B.C.D|HOST] tcp [tls [skip-host-verify]] [port (100-65535)]
Define a remote host to send syslogs. Make sure that
log syslogis enabled to make this command work. The default port is514. Users can enable TLS connection. By addingskip-host-verifyoption, the remote hostname is not checked against provided certificate CN/SAN.Example:
soodar(config)# ip host logServer 1.1.1.1 soodar(config)# log syslog logServer tcp tls
- log syslog HOST loki [skip-host-verify] [port (100-65535)]
Define a remote host to send syslogs. Make sure that
log syslogis enabled to make this command work. The default port is3000. By adding theskip-host-verifyoption, the remote hostname is not checked against the provided certificate CN/SAN in the HTTPS connection.Note
Loki connection uses
httporhttpsprotocols to communicate. User must provide thehttporhttpsin address.Note
Port is a different option. User must not provide a port in an address like
http://temp.ir:3100. It’s wrong!Example:
soodar(config)# log syslog https://192.168.1.1 loki skip-host-verify port 3100
- log monitor [LEVEL]
Enable logging output to terminal shell. By default, monitor logging is enabled at the informational level, but this command can be used to change the monitor logging level. If the optional second argument specifying the logging level is not present, the default logging level (typically informational) is used. The
noform of the command disables logging to terminal monitors.
- log facility [FACILITY]
This command changes the facility used in syslog messages. The default facility is
daemon. Thenoform of the command resets the facility to the defaultdaemonfacility.
- log record-priority
To include the severity in all messages logged to a file. Use the
log record-priorityglobal configuration command. To disable this option, use thenoform of the command. By default, the severity level is not included in logged messages.
- log timestamp precision [(0-6)]
This command sets the precision of log message timestamps to the given number of digits after the decimal point. Currently, the value must be 0 to 6 (i.e., the maximum precision is microseconds). To restore the default behavior (1-second accuracy), use the
noform of the command, or set the precision explicitly to 0.log timestamp precision 3
In this example, the precision is set to provide timestamps with millisecond accuracy.
- log commands
This command enables the logging of all commands typed by a user to all enabled log destinations.
- show log all [follow]
Show all journals logs. If the
followmode is enabled, it follows the updates.
- show log mender [follow]
Show mender update service logs. If the
followmode is enabled, it follows the updates.
- show log ssh [follow]
Show SSH service logs. If the
followmode is enabled, it follows the updates.
- show log soolog [follow]
Show Soodar service logs. We are using vector for logging. If the
followmode is enabled, it follows the updates.
- show log snmpd [follow]
Show SNMP service logs. If the
followmode is enabled, it follows the updates.
- show log ntpd [follow]
Show NTP service logs. If the
followmode is enabled, it follows the updates.
- show log vpp [follow]
Show VPP service( data plane) logs. If the
followmode is enabled, it follows the updates.
- show log frr [follow]
Show FRR service( control plane) logs. If the
followmode is enabled, it follows the updates.
- show log ipsec [follow]
Show IPSec service logs. If the
followmode is enabled, it follows the updates.
- show log kernel [follow]
Show kernel and boot logs. If the
followmode is enabled, it follows the updates.
- clear log [syslog]
Clear all generated logs. Using the
syslogkeyword makes the journald logs vacuumed; otherwise, the log file is truncated.
System update
SoodarOS uses mender as its system update solution. It supports both online and offline updates, and in case of failure, it can roll back to the previous version
Online update
Update system from a server. It is disabled by default. When an online update is enabled, the system automatically checks the server for available updates and install if any is present.
Configuration
- system update enable
Enable/Disable online update
- system update server-url WORD
Set update server’s URL
Note
Update server address, should be a URL, and an IP address can’t be set
- system update update-poll-interval (5-2147483647)
Check for update interval in seconds
- system update inventory-poll-interval (5-2147483647)
Send system inventory in intervals. Unit is in seconds
Example:
soodar(config)# system update enable soodar(config)# system update server-url https://update.soodar.ir soodar(config)# system update update-poll-interval 300 soodar(config)# system update inventory-poll-interval 400
Offlline update
Update system from removable storage. The procedure to offline update is simple. One need to:
Install an update
Reboot
Commit the update( to make it persistent) or rollback the update( in case of failure. Reboot without a commit to rollback)
Note
To use offline update, the online update should be disabled
Configuration
- system update offline list
List available updates on removable storage
Example:
n1(config)# system update offline list 1 rls-20 2 rls-21 3 rls-21.1
- system update offline install ARTIFACT
Install update from removable storage. ARTFICAT is the relative path of update file from removable storage root, without
.menderpostfix
- system update offline commit
Commit latest installed update.
Warning
During the system’s booting, no removable storage should be plugged into the router device, or the boot fails.
System backup and restore
The router is equipped with a set of backup/restore tools. User can choose to create snapshot from running-config, startup-config and PKI Private keys. but backups can only be restored to startup-config and PKI Private keys. Each snapshot is saved with a unique user-provided tag. The same tag is used to restore the snapshot. The snapshots could be stored in two ways:
To remote host and via SFTP
To local storage
Note
For keeping integrity, all snapshots are encrypted and should not be tampered.
Warning
Backing up private keys to a remote host is ill-advised and should be avoided but if it’s needed, consider further safety measures for remote snapshots and their accessibilities.
- copy <startup-config|running-config|pki> <sftp:|system:>
Create a snapshot from current startup-config and save it to a remote host. the sftp: URI could contain username, password and address of remote computer with the snapshot tag, and the system: URI contains snapshot tag. If URI is provided, all fields are shown to user for confirmation; else user is asked for required information.
Note
sftp URI is: sftp:[user]:[password]@[host]:[tag].
Note
system URI is: system:[tag].
Examples:
soodar# ! copy startup-config with full URI soodar# copy startup-config sftp:john:1234@test:backup1 Address or name of remote host [test]? Remote host user [john]? Remote host password [*****]? Destination tag [backup1]? soodar# ! copy startup-config without providing password in URI soodar# copy startup-config sftp:john@192.168.1.2:backup2 Address or name of remote host [192.168.1.2]? Remote host user [john]? Remote host password [admin]? Destination tag [backup2]? soodar# ! copy startup-config with providing only tag name soodar# copy startup-config sftp:backup3 Address or name of remote host [192.168.1.1]? Remote host user [admin]? Remote host password [admin]? Destination tag [backup3]? soodar# ! copy startup-config without providing anything soodar# copy startup-config sftp: Address or name of remote host [192.168.1.1]? Remote host user [admin]? Remote host password [admin]? Destination tag [router-config]? soodar# ! copy to system storage soodar# copy running-config system: Destination tag [router-config]? backup4
- copy <sftp:|system:> <startup-config|pki>
Restore a snapshot from provided source.
Note
restored snapshot takes effect after rebooting system.
soodar# copy system: startup-config Tag to restore [router-config]? backup4
- show archive snapshots [sftp:|system:]
List available snapshots in source.
soodar# show archive snapshots system: Tag Host Time Type ======================================================= r1 soodar Wed Jun 15 14:07:45 2022 Config ------------------------------------------------------- keybackup1 soodar Fri Jun 24 14:13:22 2022 PKI ------------------------------------------------------- backup4 soodar Sun Jul 3 14:50:37 2022 Config
- show archive config <sftp:|system:>
Show snapshot contents. only config snapshots can be shown.
soodar# show archive config system: Destination tag [router-config]? r1 r1 == hostname soodar no ipv6 forwarding no zebra nexthop kernel enable security passwords min-length 8 log syslog errors log monitor no banner motd ! no ntp ! interface ge1 no ip address ! interface ge2 no ip address ! interface ge3 no ip address ! interface lo no ip address ! interface ge0 no shutdown ip address 192.168.1.55/24 exit ! end
- show archive config differences <startup-config|running-config|sftp:|system:> <startup-config|running-config|sftp:|system:>
Compare snapshots and print the differences.
soodar# show archive config differences running-config system:r1 Destination tag [r1]? running-config r1 ============== == hostname soodar hostname soodar no ipv6 forwarding no ipv6 forwarding no zebra nexthop kernel enable no zebra nexthop kernel enable security passwords min-length 8 security passwords min-length 8 log syslog errors log syslog errors log monitor log monitor no banner motd no banner motd ! ! no ntp no ntp ! ! interface ge1 interface ge1 no ip address no ip address ! ! > interface ge2 > no ip address > ! interface ge3 interface ge3 no ip address no ip address ! ! interface lo interface lo no ip address no ip address ! ! interface ge0 interface ge0 no shutdown no shutdown ip address 192.168.1.55/24 ip address 192.168.1.55/24 exit < ! < interface ge2 < no shutdown < ip address 1.1.1.1/24 < exit exit ! ! end end
- delete <system:|sftp:>
Delete snapshot from provided source
soodar# delete sftp: Address or name of remote host [192.168.1.1]? 192.168.1.2 Remote host user [admin]? john Remote host password [admin]? Destination tag [router-config]? backup3
Prometheus Monitoring
SoodarOS supports both SNMP and Prometheus for monitoring purposes. Users can enable Prometheus monitoring by running soomon service on the router. After running and enabling soomon service, the router can provide metrics on port 9200.
- system service enable soomon
Start soomon service to provide Prometheus monitoring.
Note
Currently, soomon only works on port 9200. This behavior could change in the future.
System Services
Services are running in the background for accomplishing tasks. These services include:
NTP: Network Time Protocol service.
Mender: System update service.
Soolog: Remote and local syslog service.
SNMPD: SNMP Services
VPP: Router service. Restarting this service is like restarting the router.
soomon: Soodar Prometheus monitoring service.
- show system service status SERVICE
Show service status based on the output of systemd
- system service restart SERVICE
Restart a service. If service is not running, starts the service.
Note
An explicitly disabled service can not be restarted( for example, when a user has set
no ntpcommand, one can not restart the NTP service).
System Security
The admin can set the maximum TCP SYN limit to protect the system from SYN flood attacks.
- tcp syn-flood limit (1-4294967295)
Set TCP SYN limit. The default limit is
256.
URPF is a technique to help prevent IP Address spoofing by restricting the incoming packet’s source address to those that the router has a route to. Failed packets are discarded.
URPF has two modes of operation:
Strict: Every ingress packet’s source is looked up in the FIB; if there’s a route to source address but the incoming interface is not the output interface of the path, the packet check fails.
Loose: Every ingress packet’s source is looked up in the FIB; if there’s no route to source address by any interface, the packet is discarded.
- ip verify unicast source reachable-via [rx | any]
Enable URPF on an interface.
rxkeyword set URPF in strict mode. The default mode isany( loose mode).