Reference:
https://kb.vmware.com/s/article/68149

Symptoms

1. Unable to login with root credentials after updating vCSA 6.7 with embedded PSC to vCSA 6.7 U2 using the VAMI option.
2. Logging into the VAMI page with root credentials prompts “Invalid Credentials” message.
3. Login to the vCSA with virtual machine console works using same root credentials.

Status of services:

root@nsi-vsphere55 [ ~ ]# service-control –status
Stopped:
 applmgmt vmcam vmware-imagebuilder vmware-mbcs vmware-netdumper vmware-rbd-watchdog vmware-vcha vsan-dps
Running:
 lwsmd pschealth vmafdd vmcad vmdird vmdnsd vmonapi vmware-analytics vmware-certificatemanagement vmware-cis-license vmware-cm vmware-content-library vmware-eam vmware-perfcharts vmware-pod vmware-postgres-archiver vmware-rhttpproxy vmwa
re-sca vmware-sps vmware-sts-idmd vmware-stsd vmware-topologysvc vmware-updatemgr vmware-vapi-endpoint vmware-vmon vmware-vpostgres vmware-vpxd vmware-vpxd-svcs vmware-vsan-health vmware-vsm vsphere-client vsphere-ui vmware-statsmonitor 

Cause

1. This happens because the vmware-statsmonitor service takes a long time to start upon the vCSA 6.7 U2 reboot.
2. In this delayed duration, the applmgmt service gets marked as stopped.
3. This causes the failed logins to the VAMI page and a simple start of applmgmt service fixes it, but the behavior happens on every reboot.

Resolution

The solve the issue change the timeout for the applmgmt service so that it is started after the statsmonitor service has successfully started with its delay.

To do this, follow the below steps given below:

Go to the below directory:

cd /etc/vmware/vmware-vmon/svcCfgfiles/

Backup the applmgmt and statsmonitor json files:

mkdir /root/backup
cp /etc/vmware/vmware-vmon/svcCfgfiles/applmgmt.json /root/backup/
cp /etc/vmware/vmware-vmon/svcCfgfiles/statsmonitor.json /root/backup/

Change permissions on the existing json files:

chmod 700 applmgmt.json
chmod 700 statsmonitor.json

Manually increase the timeout in applmgmt.json file to 600 seconds(using VI editor).

The timeout in the this file is 60 seconds by default./etc/vmware/vmware-vmon/svcCfgfiles/applmgmt.json:
{
    “Name” : “applmgmt”,
    “PreStartCommand” : “/usr/lib/applmgmt/support/scripts/prestart-applmgmt.sh”,
    “StartCommand” : “/usr/lib/applmgmt/applmgmt.launcher”,
    “ApiHealthCommand”: “/usr/bin/python”,
    “ApiHealthCommandArgs”: [
        “/usr/lib/applmgmt/applmgmt_vmonhealth.py”
    ],
    “DependsOn” : [“statsmonitor”],
 “StartTimeout” : 600,
    “StopTimeout” : 20,
    “StartupType” : “AUTOMATIC”,
    “DumpLiveCoreOnApiHealthFail” : false,
    “StreamRedirectFile”: “%VMWARE_LOG_DIR%/vmware/applmgmt/applmgmt_vmonsvc”,
    “RecoveryActionProfiles” :
    {
        “DEFAULT” :
        {
            “CRASH” : [“RESTART_SERVICE”, “RESTART_SERVICE”, “NO_ACTION”],
            “HEALTHFAIL” : [“RESTART_SERVICE”, “RESTART_SERVICE”, “NO_ACTION”]
        },
        “FAILOVER” :
        {
            “CRASH” : [“RESTART_SERVICE”, “RESTART_SERVICE”, “NO_ACTION”],
            “HEALTHFAIL” : [“RESTART_SERVICE”, “RESTART_SERVICE”, “NO_ACTION”]
        }
    }
} 

Modify statsmonitor service config for vMon to set higher startup timeout:

sed -i ‘/StartTimeout/d’ /etc/vmware/vmware-vmon/svcCfgfiles/statsmonitor.json
sed -i ‘/ApiHealthFile/a “StartTimeout”: 600,’ /etc/vmware/vmware-vmon/svcCfgfiles/statsmonitor.json
This adds a StartTimeout”: 600 to the statsmonitor.json.

Stop and start statsmonitor service explicitly:

/usr/lib/vmware-vmon/vmon-cli -k statsmonitor
/usr/lib/vmware-vmon/vmon-cli -i statsmonitor

Reboot the vCSA VM.

Note: Upon reboot, all the services should be up and running in between 5-10 minutes.

After this, login to the VAMI page with root credentials should be successful.

Workaround

Run the below command to workaround this issue. However, the issue will re-occur upon a reboot.

service-control –start applmgmt