Welcome back to Check Point Jumpstart Maestro training. And in this section, we're going to look at some command line commands on both the orchestrator appliance, as well as the single management object of security groups, to examine what can you do for diagnostic troubleshooting or monitoring in the command line? It may be that you need to restart the orchestrator service itself, I'm going to time how long that takes. The command is orchd space restart. And off we go. So working on shutting down the orchestrator service. Done, now starting the orchestrator service. I point out that currently in this demo environment, there are two orchestrators. So they're doing active, active, I shut down one. So as long as things were properly redundantly cabled, then there should have been no traffic outage. But it's also possible that I may have made a configuration change to the other orchestrator while the orchestrator service on this appliance was down. So it will check to see if there have been any changes to the configuration files. And as it turns out, there hasn't, but good thing to check. And there you go, a little bit under a minute and a half for a fairly unloaded orchestrator. Clear the screen, and there are some other commands that I just want to demonstrate quickly. The orch_info command, does a pretty comprehensive diagnostic dump of log files, and config files, and other interesting things. So you would do this and upload it as part of a service request. It's very helpful or for assisting, so you definitely want to do an orch_info, again on the orchestrator. And then what you get is a gzip tar file. So there's a magic incantation which will list the contents. And it's going to be a long list. So I'm going to pipe it through my paginator. So here's a lot of log files, config files, and the output of some commands that were run, stored to the temp directory so that they would be included in this. Next, there's a config file that was included in that. In /etc/sdgb.json file, and this has the configuration of all security groups on the orchestrator appliance. So the appliances, the security gateway modules that are assigned to security groups, how many security groups there are, uplink ports assigned to the security group. So I have currently two security groups defined. And this is the second one, note in the middle of the screen there, it's a vsx security group. This file also exists on the single management object in all of the security gateway modules in a security group, but only has the contents relevant to that security group and doesn't list any data about other security groups. Another useful command And I'm going to paginate this, as it may just take up most of my screen. So this command here is dumping the state of the physical ports on the orchestrator. So which ports have something cabled in, which ports don't, etc, and useful to get just a quick picture of the settings for the ports on the orchestrator. The command lldpctl or Link Layer Discovery Protocol, I'll paginate this, this prints the information that was discovered via lldp. So we can see that there's another orchestrator out there, there are some security gateway modules. And right now, there's only two security gateway modules displayed because of an issue in the lab where the demo environment is hosted. Normally, you would have way, way more than that, but this will give you for instance, this is one way to get the serial number of the security gateways. Next I'm going to show you the member command. But you can abbreviate it, I'm a lazy typist. So you can abbreviate it m. This allows you to secure shell from the orchestrator, into any security gateway module that's up. And I guess I'll do that. So now I am connecting to the single management object because it's security gateway module one, by default it's the single management object of the first security group. So that's what the first one means, it's security group 1, the second one means member 1. And note the IP address it displays here, 198.51.101. This is an internal network that is built between the orchestrator on all security gateway modules. You can ignore the diagnostics message, they said there's an issue in the lab. So you need to be in expert mode for this command. Now, I'm in a security group, here I can't move to a different security group, I can only move to a different security gateway module. So I'm going to do that. Now I'm on security gateway module 2. And here in security gateway module 2 of security group 1, there is a copy of this sgdb.json file, but it only lists information about this security group. So the other vsx security group, security group 2 is not displayed here. So next I want to show you some more commands On the single management object, another useful command asg on a single management object has many options. One of them, asg monitor will put up a constantly updated display, by default, it refreshes every second, you can change that by specifying a different interval. But this is showing the status of this security group and the security gateway modules that are in it. There are two security gateway modules assigned to this security group and it is currently happy, the security gateway modules are active/active. To get out of this command, I usually use control C. And a slight variant of this command, I'll show you the help, there's various options. For instance you can list virtual systems if you're in a vsx security group, but the minus V option gives you a little bit different information. Here, it shows the health of various components, and over on the right, there's a weight value, which you can modify that. But for instance if a security gateway module were down, one out of two were working, leaving one that's down, then the weight of a security gateway module, which is six would be absent. So right now everything is healthy, and if you multiply everything by its weight and add all of that together you come to 56. That's the maximum possible weight when everything is working. And if one of the security gateway modules was down, then we would lose six points and we would be at 50 over 56. And that's useful for some advanced configuration multi-site deployment of orchestrator. Asg stat command gives you information about again the the security group, just in a slightly different format. There are two security gateway modules that are currently happy. Switch back to not full screen mode for this next command asg debug verify, I find it to be very useful. This command runs several diagnostics on the orchestrator and on the security group and it provides concise summary of the result of each of the diagnostics. See that some are failing, this is for various reasons such as I've not yet installed policy security group. Also my demo environment is having a bit too. This command is typically followed by the asg debug list command or for information. The dxl calc command is useful for visualizing distribution modes. And so, recall that distribution modes determine which security gateway module should handle the the traffic for a given connection. And the default is General where both the source and destination IP address and by default, layer for ports are also considered. So source IP address and port destination IP address and port are all used in the algorithm to determine which security gateway module should handle the traffic for this connection. Another option is user distribution mode, which uses the source port and destination IP. Or network distribution which uses the destination port and Source IP. We'll look a little bit more into the distribution modes coming up, another useful asg command. This shows you key performance indicators, so note the load averages that are displayed for both core Excel cores and secure Excel or secure network distributor cores. And this is a very small deployment with no policy, no traffic, so this isn't very interesting. But this command automatically updates and is useful in a production system. Next, really quickly, I wanted to show the diag list command. It provides all sorts of tests that can be done. So you use asg diag print and the test number and it will conduct that test. This is just a handy cheat sheet, so you don't have to remember the numbers. So asg diag command is very useful for running system diagnostics. Earlier, we talked about distribution modes. Distribution modes set the algorithm which determines which security gateway module should handle the traffic for a given connection. It does that by looking at either the source or destination IP address, depending on the mode chosen and source or destination port depending on if later four is enabled as part of the decision process. And so the default distribution mode is general. Which uses both the source and destination IP addresses, and source and destination ports for the decision. There's also user distribution mode, which uses the destination IP address and source port. There's the network distribution mode which uses the source IP and the destination port, and there's also topology, auto-topology mode which uses the information from the single management object. Security gateway object that was created in smart dashboard or sorry, smart console with topology indicating which interfaces are internal, which interfaces are external. It uses that. So you can see the current distribution and you need to be in global cliche show which I was an expert mode. I exited out of that back into the global cliche show. And right now the distribution mode is general and you can change that. You can show upper interface basis. Also see the status of the distribution mode. Earlier we talked a little bit about the correction layer. And again the tram of the correction layer is to handle netted traffic, because the distribution mode algorithm uses source and or destination IP, source and or destination port. To determine with this packet which security gateway module should be handling the traffic for this connection. And with NAT you're changing either the source and or destination IP address, and possibly source and or destination port. So the distribution mode algorithm will pick the wrong security gateway module. The correction layer exists to forward the traffic that was handed off to this second security getway module, back to the original security gateway module which was handling the unnetted traffic. So it sees all of the traffic which is necessary with stateful inspection. You can see some correction layer statistics, again in this demonstration environment there's really no traffic and I certainly haven't turned on that. So the correction layer hasn't had to do anything so far. So some best practices, I mean it depends on the specific network mix of the deployment site. But generally distribution mode using both source and destination IP source and destination port will do the most granular distribution of connections to the security gateway modules in the security group. However, if you're using NAT then to simplify the job of the correction layer, any hide netted networks set them to use user mode. So we're looking at the destination IP and source port. And the destination networks, if for instance static NAT you would use network mode which uses the source IP in the destination port in the distribution algorithm. The global cliche show is very convenient in a security group because the configuration changes that you make in global cliche show are automatically propagated to the other security gateway modules in the security group. But in expert mode, There are a number of command line global commands that begin with g_. So 1g_ command that's useful, g_update_conf_file. And so this command wants the path to a config file and a var equal value pair. Though if you leave off the value, it will just remove that var from the config file. So you obviously don't want to do this unless you have a very good reason to be editing a config file. But for an example this fwkern config file, I'm going to change, The arp forwarding setting, rathet than risk typo it. Copy paste, And you can see that the value has been changed in the config file on this security gateway module. You can ignore that build system diagnostics background issue. You can see that the value has been changed here as well. Well I'm going to go ahead and put it back. And see that it's been changed on the second security gateway module in the security group. Going back to the first security gateway module, you can see it's been changed back. So useful command if you need to update some config file parameter on all of the security gateway modules in your security group. Another useful command is a global TCP dump command. So same syntax as TCP dump, it will aggregate all of the packet capture data and very useful for getting a large amount of packet capture data fairly quickly. You probably want to use capture filters. Generically the g_all command will run whatever command you give it as an argument. It will run that on all of the security gateway modules. So Come up with a command. That'll possibly someday be useful. Show me the tail end of the same config file fwkern.conf. And only two lines in that file, that was it a hard thing for Tail to do, but it ran it on all of the security gateway modules in the security group and then showed the output by security gateway module here. If you have a security group defined you automatically get active backup clustering. There will be one security gateway module that is designated active, and then one that is designated backup in the same security group. So what happens with packets? The orchestrator receives a packet from an Uplink interface. It uses the distribution mode algorithm to calculate which port, downlink port that connection should be handled by, and the algorithm guarantees that it will always be a downlink port that security gateway module is plugged into. So it'll calculate that it needs to go out this port, it'll switch the packet out that port it'll be received by the security gateway module. And that security gateway module if this is a new connection, will calculate which other security gateway module in the security group should be back up and start doing state synchronization to that security gateway module and that's done over the synchronization VLAN. So you have the active that's processing the traffic for the connection and the back up that's being synchronized, so it has up-to-date state table such as connections table information. And so because there's only two security gateway modules that are participating in the clustering, you don't have the extra overhead that the general products cluster Excel experiences. The Maestro, I guess, version of clustering, they call it Hypersync, limits overhead and so there's a performance penalty of about 1% of throughput per security gateway module in the security group. So if you have four security gateway modules in the security group, then you are getting, roughly, 96 percent of the aggregate throughput capability of those four security gateway modules. There's some overhead. And this is much much better than cluster Excel load sharing where in my experience when you get to maybe five active active load sharing gateways in the cluster, adding any more makes it worse because of all the overhead of synchronizing everybody to everybody. By limiting the synchronization between the active and the backup that really eliminates a lot of the overhead of the synchronization and again, it's about 1% penalty. So that wraps up the command line and the little bit of troubleshooting that I wanted to show you. Thank you very much for attending this jump start training on the Maestro Hyperscale network security solution.
