Hey there,
since the last update im having several problems, one particular with command issuing and echo timeout i guess.
While issuing the following commands in a job
sh version
failover exec mate sh vers
show inventory
failover exec mate sh inventory
show int | grep error
sh access-list outside-access-out | i line 1 extended
sh access-list outside_access_out | i line 1 extended
sh failover
2 of about 150 devices are aborting due to command error and just showing a single line of the first command in the file:
asa-emft-oph# sh version
the same when issuing just "sh run"
Here the debug output for those 2 devices:
Device1:
<W-10:37:50>ssh -2 -l nocadmin XXXXXXXXX
<R-10:37:50>ssh -2 -l nocadmin XXXXXXXXXXX
<W-10:37:50>[13]
<R-10:37:50>[13][10]
<R-10:37:50>nocadmin@XXXXXXX's password:
<W-10:37:50>XXXXXXXXXXXXXXXXX[13]
<R-10:37:50>[13][10]
<R-10:37:51>Type help or '?' for a list of available commands.[13][10][13]asa-emft-oph#
<W-10:37:51>[13]
<W-10:37:51>[13]
<R-10:37:51>[13][10][13]asa-emft-oph#
<W-10:37:51>[13]
<R-10:37:51>[13][10][13]asa-emft-oph#
<W-10:37:51>sh version
<R-10:37:51>[13][10][13]asa-emft-oph#
<R-10:37:51>sh version
<W-10:37:51>[13]
<W-10:37:52>failover exec mate sh vers
<R-10:37:52>[13][10][13][10]Cisco Adaptive Security Appliance Software Version 9.1(7)23 [13][10]Device Manager Version 7.8(1)[13][10][13][10]Compiled on Thu 01-Feb-18 23:08 by builders[13][10]System image file is "disk0:/asa917-23-k8.bin"[13][10]Config file at boot was "disk0:/asa-emft-oph.cfg"[13][10][13][10]asa-emft-oph up 125 days 4 hours[13][10][13][10]Hardware: ASA5520, 2048 MB RAM, CPU Pentium 4 Celeron 2000 MHz,[13][10]Internal ATA Compact Flash, 256MB[13][10]Slot 1: ATA Compact Flash, 512MB[13][10]BIOS Flash M50FW016 @ 0xfff00000, 2048KB[13][10][13][10]Encryption hardware device : Cisco ASA-55xx on-board accelerator (revision 0x0)[13][10] Boot microcode : CN1000-MC-BOOT-2.00 [13][10] SSL/IKE microcode : CNlite-MC-SSLm-PLUS-2.08[13][10] IPSec microcode : CNlite-MC-IPSECm-MAIN-2.09[13][10] Number of accelerators: 1[13][10][13][10] 0: Ext: GigabitEthernet0/0 : address is f866.f2c4.a4cc, irq 9[13][10] 1: Ext: GigabitEthernet0/1 : address is f866.f2c4.a4cd, irq 9[13][10] 2: Ext: GigabitEthernet0/2 : address is f866.f2c4.a4ce, irq 9[13][10] 3: Ext: GigabitEthernet0/3 : address is f866.f2c4.a4cf, irq 9[13][10]<--- More --->
<R-10:37:52>[13] [13] 4: Ext: Management0/0 : address is f866.f2c4.a4cb, irq 11[13][10] 5: Int: Internal-Data0/0 : address is 0000.0001.0002, irq 11[13][10] 6: Int: Internal-Control0/0 : address is 0000.0001.0001, irq 5[13][10][13][10]Licensed features for this platform:[13][10]Maximum Physical Interfaces : Unlimited perpetual[13][10]Maximum VLANs : 150 perpetual[13][10]Inside Hosts : Unlimited perpetual[13][10]Failover : Active/Active perpetual[13][10]Encryption-DES : Enabled perpetual[13][10]Encryption-3DES-AES : Enabled perpetual[13][10]Security Contexts : 2 perpetual[13][10]GTP/GPRS : Disabled perpetual[13][10]AnyConnect Premium Peers : 2 perpetual[13][10]AnyConnect Essentials : Disabled perpetual[13][10]Other VPN Peers : 750 perpetual[13][10]Total VPN Peers : 750 perpetual[13][10]Shared License : Disabled perpetual[13][10]AnyConnect for Mobile : Disabled perpetual[13][10]AnyConnect for Cisco VPN Phone : Disabled perpetual[13][10]Advanced Endpoint Assessment : Disabled perpetual[13][10]UC Phone Proxy Sessions : 2 perpetual[13][10]Total UC Proxy Sessions : 2 perpetual[13][10]Botnet Traffic Filter : Disabled perpetual[13][10]<--- More --->[13] [13]Intercompany Media Engine : Disabled perpetual[13][10]Cluster : Disabled perpetual[13][10][13][10]This platform has an ASA 5520 VPN Plus license.[13][10][13][10]Serial Number: JMX1447L037[13][10]Running Permanent Activation Key: 0xd827ec71 0x88d7e59b 0x9c212148 0x9d108494 0x8f201181 [13][10]
<R-10:37:53>Configuration register is 0x1[13][10]Configuration last modified by admin at 13:33:40.297 CEDT Wed Jun 13 2018[13][10][13]asa-emft-oph# mate sh vers
================================================================================
WFDRetVal=0. Waiting for: "failoverexecmateshvers"
WFDBuffer="ciscoadaptivesecurityappliancesoftwareversion9.1(7)23devicemanagerversion7.8(1)compiledonthu01-feb-1823:08bybuilderssystemimagefileis"disk0:/asa917-23-k8.bin"configfileatbootwas"disk0:/asa-emft-oph.cfg"asa-emft-ophup125days4hourshardware:asa5520,2048mbram,cpupentium4celeron2000mhz,internalatacompactflash,256mbslot1:atacompactflash,512mbbiosflashm50fw016@0xfff00000,2048kbencryptionhardwaredevice:ciscoasa-55xxon-boardaccelerator(revision0x0)bootmicrocode:cn1000-mc-boot-2.00ssl/ikemicrocode:cnlite-mc-sslm-plus-2.08ipsecmicrocode:cnlite-mc-ipsecm-main-2.09numberofaccelerators:10:ext:gigabitethernet0/0:addressisf866.f2c4.a4cc,irq91:ext:gigabitethernet0/1:addressisf866.f2c4.a4cd,irq92:ext:gigabitethernet0/2:addressisf866.f2c4.a4ce,irq93:ext:gigabitethernet0/3:addressisf866.f2c4.a4cf,irq9<---more--->4:ext:management0/0:addressisf866.f2c4.a4cb,irq115:int:internal-data0/0:addressis0000.0001.0002,irq116:int:internal-control0/0:addressis0000.0001.0001,irq5licensedfeaturesforthisplatform:maximumphysicalinterfaces:unlimitedperpetualmaximumvlans:150perpetualinsidehosts:unlimitedperpetualfailover:active/activeperpetualencryption-des:enabledperpetualencryption-3des-aes:enabledperpetualsecuritycontexts:2perpetualgtp/gprs:disabledperpetualanyconnectpremiumpeers:2perpetualanyconnectessentials:disabledperpetualothervpnpeers:750perpetualtotalvpnpeers:750perpetualsharedlicense:disabledperpetualanyconnectformobile:disabledperpetualanyconnectforciscovpnphone:disabledperpetualadvancedendpointassessment:disabledperpetualucphoneproxysessions:2perpetualtotalucproxysessions:2perpetualbotnettrafficfilter:disabledperpetual<---more--->intercompanymediaengine:disabledperpetualcluster:disabledperpetualthisplatformhasanasa5520vpnpluslicense.serialnumber:jmx1447l037runningpermanentactivationkey:0xd827ec710x88d7e59b0x9c2121480x9d1084940x8f201181configurationregisteris0x1configurationlastmodifiedbyadminat13:33:40.297cedtwedjun132018asa-emft-oph#mateshvers"
================================================================================
<W-10:38:23>[13]
<R-10:38:23>[13][10] ^[13][10]ERROR: % Invalid input detected at '^' marker.[13][10][13]asa-emft-oph#
<W-10:38:23>[13]
<R-10:38:23>[13][10][13]asa-emft-oph#
<W-10:38:23>disable
<R-10:38:24>disable
<W-10:38:24>[13]
<R-10:38:24>[13][10][13]asa-emft-oph>
<W-10:38:24>[13]
<R-10:38:24>[13][10][13]asa-emft-oph>
<W-10:38:24>exit[13]
<D 10:38:24>
<SCRIPT VALUES>
<HOSTNAME="asa-emft-oph">
<PROMPT VTY="asa-emft-oph>">
<PROMPT ENABLE="asa-emft-oph#">
<PROMPT CONFIG="asa-emft-oph(">
Device2:
<W-10:37:50>ssh -2 -l nocadmin XXXXXXXX
<R-10:37:50>ssh -2 -l nocadmin XXXXXXXXXX
<W-10:37:50>[13]
<R-10:37:50>[13][10]
<R-10:37:50>Unauthorized access prohibited[13][10]nocadmin@192.44.23.130's password:
<W-10:37:50>XXXXXXXXXXXXXX[13]
<R-10:37:50>[13][10]
<R-10:37:51>User nocadmin logged in to asa-iml[13][10]Logins over the last 127 days: 521. Last login: 10:13:13 CEDT Jun 14 2018 from 153.96.2.16[13][10]Failed logins since the last login: 0. [13][10]Type help or '?' for a list of available commands.[13][10][13]asa-iml#
<W-10:37:51>[13]
<W-10:37:51>[13]
<R-10:37:51>[13][10][13]asa-iml#
<W-10:37:51>[13]
<R-10:37:51>[13][10][13]asa-iml#
<W-10:37:51>sh version
<R-10:37:51>[13][10][13]asa-iml#
<R-10:37:51>sh version
<W-10:37:51>[13]
<W-10:37:51>failover exec mate sh vers
<R-10:37:51>[13][10][13][10]Cisco Adaptive Security Appliance Software Version 9.8(2)20 [13][10]Firepower Extensible Operating System Version 2.2(2.63)[13][10]Device Manager Version 7.8(1)[13][10][13][10]Compiled on Fri 02-Feb-18 06:18 PST by builders[13][10]System image file is "disk0:/asa982-20-smp-k8.bin"[13][10]Config file at boot was "disk0:/asa-iml.cfg"[13][10][13][10]asa-iml up 126 days 19 hours[13][10]failover cluster up 302 days 16 hours[13][10][13][10]Hardware: ASA5545, 12288 MB RAM, CPU Lynnfield 2660 MHz, 1 CPU (8 cores)[13][10] ASA: 6466 MB RAM, 1 CPU (1 core)[13][10]Internal ATA Compact Flash, 8192MB[13][10]BIOS Flash MX25L6445E @ 0xffbb0000, 8192KB[13][10][13][10]Encryption hardware device : Cisco ASA Crypto on-board accelerator (revision 0x1)[13][10] Boot microcode : CNPx-MC-BOOT-2.00[13][10] SSL/IKE microcode : CNPx-MC-SSL-SB-PLUS-0005[13][10] IPSec microcode : CNPx-MC-IPSEC-MAIN-0026[13][10] Number of accelerators: 1[13][10]Baseboard Management Controller (revision 0x1) Firmware Version: 2.4[13][10][13][10][13][10]<--- More --->
<R-10:37:51>[13] [13] 0: Int: Internal-Data0/0 : address is 0027.e3e4.0220, irq 11[13][10] 1: Ext: GigabitEthernet0/0 : address is 0027.e3e4.0225, irq 5[13][10] 2: Ext: GigabitEthernet0/1 : address is 0027.e3e4.0221, irq 5[13][10] 3: Ext: GigabitEthernet0/2 : address is 0027.e3e4.0226, irq 10[13][10] 4: Ext: GigabitEthernet0/3 : address is 0027.e3e4.0222, irq 10[13][10] 5: Ext: GigabitEthernet0/4 : address is 0027.e3e4.0227, irq 5[13][10] 6: Ext: GigabitEthernet0/5 : address is 0027.e3e4.0223, irq 5[13][10] 7: Ext: GigabitEthernet0/6 : address is 0027.e3e4.0228, irq 10[13][10] 8: Ext: GigabitEthernet0/7 : address is 0027.e3e4.0224, irq 10[13][10] 9: Int: Internal-Data0/1 : address is 0000.0001.0002, irq 0[13][10]10: Int: Internal-Control0/0 : address is 0000.0001.0001, irq 0[13][10]11: Int: Internal-Data0/2 : address is 0000.0001.0003, irq 0[13][10]12: Ext: Management0/0 : address is 0027.e3e4.0220, irq 0[13][10]13: Int: Internal-Data0/3 : address is 0000.0100.0001, irq 0[13][10][13][10]Licensed features for this platform:[13][10]Maximum Physical Interfaces : Unlimited perpetual[13][10]Maximum VLANs : 300 perpetual[13][10]Inside Hosts : Unlimited perpetual[13][10]Failover : Active/Active perpetual[13][10]Encryption-DES : Enabled perpetual[13][10]Encryption-3DES-AES : Enabled perpetual[13][10]Security Contexts : 2 perpetual[13][10]Carrier : Disabled perpetual[13][10]<--- More --->[13] [13]AnyConnect Premium Peers : 2500 perpetual[13][10]AnyConnect Essentials : Disabled perpetual[13][10]Other VPN Peers : 2500 perpetual[13][10]Total VPN Peers : 2500 perpetual[13][10]AnyConnect for Mobile : Enabled perpetual[13][10]AnyConnect for Cisco VPN Phone : Enabled perpetual[13][10]Advanced Endpoint Assessment : Enabled perpetual[13][10]Shared License : Disabled perpetual[13][10]Total TLS Proxy Sessions : 2 perpetual[13][10]Botnet Traffic Filter : Disabled perpetual[13][10]IPS Module : Disabled perpetual[13][10]Cluster : Enabled perpetual[13][10]Cluster Members : 2 perpetual[13][10][13][10]This platform has an ASA5545 VPN Premium license.[13][10][13][10][13][10]Failover cluster licensed features for this platform:[13][10]Maximum Physical Interfaces : Unlimited perpetual[13][10]Maximum VLANs : 300 perpetual[13][10]Inside Hosts : Unlimited perpetual[13][10]Failover : Active/Active perpetual[13][10]Encryption-DES : Enabled perpetual[13][10]Encryption-3DES-AES : Enabled perpetual[13][10]<--- More --->
<R-10:37:51>[13] [13]Security Contexts : 4 perpetual[13][10]Carrier : Disabled perpetual[13][10]AnyConnect Premium Peers : 2500 perpetual[13][10]AnyConnect Essentials : Disabled perpetual[13][10]Other VPN Peers : 2500 perpetual[13][10]Total VPN Peers : 2500 perpetual[13][10]AnyConnect for Mobile : Enabled perpetual[13][10]AnyConnect for Cisco VPN Phone : Enabled perpetual[13][10]Advanced Endpoint Assessment : Enabled perpetual[13][10]Shared License : Disabled perpetual[13][10]Total TLS Proxy Sessions : 4 perpetual[13][10]Botnet Traffic Filter : Disabled perpetual[13][10]IPS Module : Disabled perpetual[13][10]Cluster : Enabled perpetual[13][10][13][10]This platform has an ASA5545 VPN Premium license.[13][10][13][10]Serial Number: FCH21147VP5[13][10]Running Permanent Activation Key: 0x752ec867 0x24c86c51 0x11727950 0xda9cf0d8 0xc01cf2bc [13][10]
<R-10:37:52>Configuration register is 0x1[13][10][13][10]Image type : Release[13][10]Key version : A[13][10][13][10]<--- More --->[13] [13]Configuration last modified by root at 09:56:02.322 CEDT Wed Jun 13 2018[13][10][13]asa-iml# vers
================================================================================
WFDRetVal=0. Waiting for: "failoverexecmateshvers"
WFDBuffer="ciscoadaptivesecurityappliancesoftwareversion9.8(2)20firepowerextensibleoperatingsystemversion2.2(2.63)devicemanagerversion7.8(1)compiledonfri02-feb-1806:18pstbybuilderssystemimagefileis"disk0:/asa982-20-smp-k8.bin"configfileatbootwas"disk0:/asa-iml.cfg"asa-imlup126days19hoursfailoverclusterup302days16hourshardware:asa5545,12288mbram,cpulynnfield2660mhz,1cpu(8cores)asa:6466mbram,1cpu(1core)internalatacompactflash,8192mbbiosflashmx25l6445e@0xffbb0000,8192kbencryptionhardwaredevice:ciscoasacryptoon-boardaccelerator(revision0x1)bootmicrocode:cnpx-mc-boot-2.00ssl/ikemicrocode:cnpx-mc-ssl-sb-plus-0005ipsecmicrocode:cnpx-mc-ipsec-main-0026numberofaccelerators:1baseboardmanagementcontroller(revision0x1)firmwareversion:2.4<---more--->0:int:internal-data0/0:addressis0027.e3e4.0220,irq111:ext:gigabitethernet0/0:addressis0027.e3e4.0225,irq52:ext:gigabitethernet0/1:addressis0027.e3e4.0221,irq53:ext:gigabitethernet0/2:addressis0027.e3e4.0226,irq104:ext:gigabitethernet0/3:addressis0027.e3e4.0222,irq105:ext:gigabitethernet0/4:addressis0027.e3e4.0227,irq56:ext:gigabitethernet0/5:addressis0027.e3e4.0223,irq57:ext:gigabitethernet0/6:addressis0027.e3e4.0228,irq108:ext:gigabitethernet0/7:addressis0027.e3e4.0224,irq109:int:internal-data0/1:addressis0000.0001.0002,irq010:int:internal-control0/0:addressis0000.0001.0001,irq011:int:internal-data0/2:addressis0000.0001.0003,irq012:ext:management0/0:addressis0027.e3e4.0220,irq013:int:internal-data0/3:addressis0000.0100.0001,irq0licensedfeaturesforthisplatform:maximumphysicalinterfaces:unlimitedperpetualmaximumvlans:300perpetualinsidehosts:unlimitedperpetualfailover:active/activeperpetualencryption-des:enabledperpetualencryption-3des-aes:enabledperpetualsecuritycontexts:2perpetualcarrier:disabledperpetual<---more--->anyconnectpremiumpeers:2500perpetualanyconnectessentials:disabledperpetualothervpnpeers:2500perpetualtotalvpnpeers:2500perpetualanyconnectformobile:enabledperpetualanyconnectforciscovpnphone:enabledperpetualadvancedendpointassessment:enabledperpetualsharedlicense:disabledperpetualtotaltlsproxysessions:2perpetualbotnettrafficfilter:disabledperpetualipsmodule:disabledperpetualcluster:enabledperpetualclustermembers:2perpetualthisplatformhasanasa5545vpnpremiumlicense.failoverclusterlicensedfeaturesforthisplatform:maximumphysicalinterfaces:unlimitedperpetualmaximumvlans:300perpetualinsidehosts:unlimitedperpetualfailover:active/activeperpetualencryption-des:enabledperpetualencryption-3des-aes:enabledperpetual<---more--->securitycontexts:4perpetualcarrier:disabledperpetualanyconnectpremiumpeers:2500perpetualanyconnectessentials:disabledperpetualothervpnpeers:2500perpetualtotalvpnpeers:2500perpetualanyconnectformobile:enabledperpetualanyconnectforciscovpnphone:enabledperpetualadvancedendpointassessment:enabledperpetualsharedlicense:disabledperpetualtotaltlsproxysessions:4perpetualbotnettrafficfilter:disabledperpetualipsmodule:disabledperpetualcluster:enabledperpetualthisplatformhasanasa5545vpnpremiumlicense.serialnumber:fch21147vp5runningpermanentactivationkey:0x752ec8670x24c86c510x117279500xda9cf0d80xc01cf2bcconfigurationregisteris0x1imagetype:releasekeyversion:a<---more--->configurationlastmodifiedbyrootat09:56:02.322cedtwedjun132018asa-iml#vers"
================================================================================
<W-10:38:22>[13]
<R-10:38:22>[13][10] ^[13][10]ERROR: % Invalid input detected at '^' marker.[13][10][13]asa-iml#
<W-10:38:22>[13]
<R-10:38:22>[13][10][13]asa-iml#
<W-10:38:22>disable
<R-10:38:22>disable
<W-10:38:22>[13]
<R-10:38:22>[13][10][13]asa-iml>
<W-10:38:22>[13]
<R-10:38:22>[13][10][13]asa-iml>
<W-10:38:23>exit[13]
<D 10:38:23>
<SCRIPT VALUES>
<HOSTNAME="asa-iml">
<PROMPT VTY="asa-iml>">
<PROMPT ENABLE="asa-iml#">
<PROMPT CONFIG="asa-iml(">
The Devices are accessed with a jumphost based on linux which never made any problems before, and does not interfere with alle the other ASAs where its working, so i guess this isnt the problem.
Any suggestions?