In my previous post vCenter Server 6.* Upgrade I described process of upgrading vCenter Server 6.*. Right after upgrade I noticed that I can’t connect to vCenter Server using vSphere Web Client. It was a bit odd behaviour but I wanted to fix it.
Error I saw in browser was the following:
1
|
503 Service Unavailable (Failed to connect to endpoint: [class Vmacore::Http::LocalServiceSpec:00000000025053F0] _serverNamespace = /vsphere-client _isRedirect = false _port = 9090)
|
Here is the wirgo.log located in C:\ProgramData\VMware\vCenterServer\logs\vsphere-client
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
|
STATUS | wrapper | 2016/03/01 12:46:28 | --< Wrapper Started as Service
STATUS | wrapper | 2016/03/01 12:46:28 | Java Service Wrapper Professional Edition 64-bit 3.5.24
STATUS | wrapper | 2016/03/01 12:46:28 | Copyright (C) 1999-2014 Tanuki Software, Ltd. All Rights Reserved.
STATUS | wrapper | 2016/03/01 12:46:28 | http://wrapper.tanukisoftware.com
STATUS | wrapper | 2016/03/01 12:46:28 | Licensed to VMware Global, Inc. for VMware vSphere Web Client (custom)
STATUS | wrapper | 2016/03/01 12:46:28 |
INFO | wrapper | 2016/03/01 12:46:28 | Event Command 'wrapper_start': Command line: "C:\Program Files\VMware\vCenter Server\visl-integration\usr\sbin\cloudvm-ram-size.bat" -J vsphere-client -O "C:\ProgramData\VMware\vCenterServer\cfg/java/vsphere-client_jvm.conf"
INFO | wrapper | 2016/03/01 12:46:28 | Event Command 'wrapper_start': Command launched (pid: 5908), blocking for up to 3600 seconds...
INFO | wrapper | 2016/03/01 12:46:28 | Event Command 'wrapper_start': Command completed with exit code: 0
INFO | wrapper | 2016/03/01 12:46:28 | Event Command 'wrapper_start': Command completed with exit code: 0 Continuing.
STATUS | wrapper | 2016/03/01 12:46:29 | Launching a JVM...
INFO | jvm 1 | 2016/03/01 12:46:29 | WrapperManager: Initializing...
INFO | jvm 1 | 2016/03/01 12:46:29 | VKS keystore loaded
INFO | jvm 1 | 2016/03/01 12:46:29 | Deleted the old JKS file.
INFO | jvm 1 | 2016/03/01 12:46:29 | JKS keystore populated successfully.
INFO | jvm 1 | 2016/03/01 12:46:29 | JKS keystore written to file system successfully.
INFO | jvm 1 | 2016/03/01 12:46:29 | Tomcat config file populated successfully.
INFO | jvm 1 | 2016/03/01 12:46:32 | [2016-03-01T12:46:32.176+01:00] [INFO ] startup-tracker Kernel starting.
INFO | jvm 1 | 2016/03/01 12:46:32 | [2016-03-01T12:46:32.554+01:00] [ERROR] kernel-dm-4 Application context creation failure for bundle 'org.eclipse.virgo.kernel.services' version '3.6.3.RELEASE'. org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'kernelWorkArea' defined in URL [bundleentry://34.fwk649660992/META-INF/spring/io-context.xml]: Instantiation of bean failed; nested exception is org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [org.eclipse.virgo.kernel.services.work.StandardWorkArea]: Constructor threw exception; nested exception is org.eclipse.virgo.util.io.FatalIOException: Unable to create directory C:\PROGRA~1\VMware\VCENTE~1\WEBCLI~1\server\work\services
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:288)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1035)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:939)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:485)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:456)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:294)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:225)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:291)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:193)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:585)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:913)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.eclipse.gemini.blueprint.context.support.AbstractDelegatedExecutionApplicationContext.access$1600(AbstractDelegatedExecutionApplicationContext.java:60)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.eclipse.gemini.blueprint.context.support.AbstractDelegatedExecutionApplicationContext$4.run(AbstractDelegatedExecutionApplicationContext.java:325)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.eclipse.gemini.blueprint.util.internal.PrivilegedUtils.executeWithCustomTCCL(PrivilegedUtils.java:85)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.eclipse.gemini.blueprint.context.support.AbstractDelegatedExecutionApplicationContext.completeRefresh(AbstractDelegatedExecutionApplicationContext.java:290)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.eclipse.gemini.blueprint.extender.internal.dependencies.startup.DependencyWaiterApplicationContextExecutor$CompleteRefreshTask.run(DependencyWaiterApplicationContextExecutor.java:137)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.eclipse.virgo.kernel.agent.dm.ContextPropagatingTaskExecutor$2.run(ContextPropagatingTaskExecutor.java:95)
INFO | jvm 1 | 2016/03/01 12:46:32 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
INFO | jvm 1 | 2016/03/01 12:46:32 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
INFO | jvm 1 | 2016/03/01 12:46:32 | at java.lang.Thread.run(Thread.java:745)
INFO | jvm 1 | 2016/03/01 12:46:32 | Caused by: org.springframework.beans.BeanInstantiationException: Could not instantiate bean class [org.eclipse.virgo.kernel.services.work.StandardWorkArea]: Constructor threw exception; nested exception is org.eclipse.virgo.util.io.FatalIOException: Unable to create directory C:\PROGRA~1\VMware\VCENTE~1\WEBCLI~1\server\work\services
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:162)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:110)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:280)
INFO | jvm 1 | 2016/03/01 12:46:32 | ... 19 common frames omitted
INFO | jvm 1 | 2016/03/01 12:46:32 | Caused by: org.eclipse.virgo.util.io.FatalIOException: Unable to create directory C:\PROGRA~1\VMware\VCENTE~1\WEBCLI~1\server\work\services
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.eclipse.virgo.util.io.PathReference.createDirectory(PathReference.java:382)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.eclipse.virgo.kernel.services.work.StandardWorkArea.(StandardWorkArea.java:46)
INFO | jvm 1 | 2016/03/01 12:46:32 | at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
INFO | jvm 1 | 2016/03/01 12:46:32 | at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
INFO | jvm 1 | 2016/03/01 12:46:32 | at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
INFO | jvm 1 | 2016/03/01 12:46:32 | at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
INFO | jvm 1 | 2016/03/01 12:46:32 | at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:147)
INFO | jvm 1 | 2016/03/01 12:46:32 | ... 21 common frames omitted
|
In the logs there is one entry that is very interesting.
1
|
NFO | jvm 1 | 2016/03/01 12:46:32 | Caused by: org.eclipse.virgo.util.io.FatalIOException: Unable to create directory C:\PROGRA~1\VMware\VCENTE~1\WEBCLI~1\server\work\services
|
vSphere Web Client was running as NT Service\vspherewebclientsvc and it didn’t have correct permissions in C:\Program Files\VMware\vCenter Server\WebClient\server\work directory. Adding NT Service with full rights access to folder didn’t help as well. I changed it to main domain account (it could by any account with rights logon as a service and local admin rights on vCenter Server) and restarted vSphere Web Client.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
|
STATUS | wrapper | 2016/03/01 13:05:48 | --< Wrapper Started as Service
STATUS | wrapper | 2016/03/01 13:05:48 | Java Service Wrapper Professional Edition 64-bit 3.5.24
STATUS | wrapper | 2016/03/01 13:05:48 | Copyright (C) 1999-2014 Tanuki Software, Ltd. All Rights Reserved.
STATUS | wrapper | 2016/03/01 13:05:48 | http://wrapper.tanukisoftware.com
STATUS | wrapper | 2016/03/01 13:05:48 | Licensed to VMware Global, Inc. for VMware vSphere Web Client (custom)
STATUS | wrapper | 2016/03/01 13:05:48 |
INFO | wrapper | 2016/03/01 13:05:48 | Event Command 'wrapper_start': Command line: "C:\Program Files\VMware\vCenter Server\visl-integration\usr\sbin\cloudvm-ram-size.bat" -J vsphere-client -O "C:\ProgramData\VMware\vCenterServer\cfg/java/vsphere-client_jvm.conf"
INFO | wrapper | 2016/03/01 13:05:48 | Event Command 'wrapper_start': Command launched (pid: 9836), blocking for up to 3600 seconds...
INFO | wrapper | 2016/03/01 13:05:48 | Event Command 'wrapper_start': Command completed with exit code: 0
INFO | wrapper | 2016/03/01 13:05:48 | Event Command 'wrapper_start': Command completed with exit code: 0 Continuing.
STATUS | wrapper | 2016/03/01 13:05:48 | Launching a JVM...
INFO | jvm 1 | 2016/03/01 13:05:48 | WrapperManager: Initializing...
INFO | jvm 1 | 2016/03/01 13:05:48 | WrapperManager: Port 31,550 already in use, using port 31,551 instead.
INFO | jvm 1 | 2016/03/01 13:05:49 | VKS keystore loaded
INFO | jvm 1 | 2016/03/01 13:05:49 | Deleted the old JKS file.
INFO | jvm 1 | 2016/03/01 13:05:49 | JKS keystore populated successfully.
INFO | jvm 1 | 2016/03/01 13:05:49 | JKS keystore written to file system successfully.
INFO | jvm 1 | 2016/03/01 13:05:49 | Tomcat config file populated successfully.
INFO | jvm 1 | 2016/03/01 13:05:51 | [2016-03-01T13:05:51.798+01:00] [INFO ] startup-tracker <KE0001I< Kernel starting.
STATUS | wrapperm | 2016/03/01 13:05:55 | The VMware vSphere Web Client Service is installed.
STATUS | wrapperm | 2016/03/01 13:05:55 | Start Type: Automatic
STATUS | wrapperm | 2016/03/01 13:05:55 | Interactive: No
STATUS | wrapperm | 2016/03/01 13:05:55 | Running: Yes
INFO | jvm 1 | 2016/03/01 13:05:57 | [2016-03-01T13:05:57.668+01:00] [INFO ] startup-tracker <KE0002I< Kernel started.
INFO | jvm 1 | 2016/03/01 13:05:58 | [2016-03-01T13:05:58.014+01:00] [INFO ] system-artifacts <DE0000I< Installing plan 'org.eclipse.virgo.kernel.userregion.blueprint' version '3.6.3'.
INFO | jvm 1 | 2016/03/01 13:05:58 | [2016-03-01T13:05:58.209+01:00] [INFO ] system-artifacts <DE0000I< Installing bundle 'org.eclipse.gemini.blueprint.core' version '1.0.2.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:05:58 | [2016-03-01T13:05:58.267+01:00] [INFO ] system-artifacts <DE0000I< Installing bundle 'org.eclipse.gemini.blueprint.extender' version '1.0.2.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:05:58 | [2016-03-01T13:05:58.283+01:00] [INFO ] system-artifacts <DE0000I< Installing bundle 'org.eclipse.gemini.blueprint.io' version '1.0.2.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:05:58 | [2016-03-01T13:05:58.294+01:00] [INFO ] system-artifacts <DE0000I< Installing bundle 'org.eclipse.virgo.kernel.agent.dm' version '3.6.3.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:05:58 | [2016-03-01T13:05:58.302+01:00] [INFO ] system-artifacts <DE0000I< Installing bundle 'org.eclipse.virgo.kernel.deployer.dm' version '3.6.3.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:05:58 | [2016-03-01T13:05:58.337+01:00] [INFO ] system-artifacts <DE0000I< Installing bundle 'org.eclipse.equinox.ds' version '1.4.0.v20120112-1400'.
INFO | jvm 1 | 2016/03/01 13:05:58 | [2016-03-01T13:05:58.368+01:00] [INFO ] system-artifacts <DE0000I< Installing bundle 'org.eclipse.equinox.util' version '1.0.300.v20111010-1614'.
INFO | jvm 1 | 2016/03/01 13:05:58 | [2016-03-01T13:05:58.373+01:00] [INFO ] system-artifacts <DE0000I< Installing configuration 'osgi.console' version '0.0.0'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.284+01:00] [INFO ] system-artifacts <DE0001I< Installed bundle 'org.eclipse.gemini.blueprint.core' version '1.0.2.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.285+01:00] [INFO ] system-artifacts <DE0001I< Installed bundle 'org.eclipse.gemini.blueprint.extender' version '1.0.2.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.286+01:00] [INFO ] system-artifacts <DE0001I< Installed bundle 'org.eclipse.gemini.blueprint.io' version '1.0.2.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.287+01:00] [INFO ] system-artifacts <DE0001I< Installed bundle 'org.eclipse.virgo.kernel.agent.dm' version '3.6.3.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.293+01:00] [INFO ] system-artifacts <DE0001I< Installed bundle 'org.eclipse.virgo.kernel.deployer.dm' version '3.6.3.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.294+01:00] [INFO ] system-artifacts <DE0001I< Installed bundle 'org.eclipse.equinox.ds' version '1.4.0.v20120112-1400'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.295+01:00] [INFO ] system-artifacts <DE0001I< Installed bundle 'org.eclipse.equinox.util' version '1.0.300.v20111010-1614'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.296+01:00] [INFO ] system-artifacts <DE0001I< Installed configuration 'osgi.console' version '0.0.0'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.296+01:00] [INFO ] system-artifacts <DE0001I< Installed plan 'org.eclipse.virgo.kernel.userregion.blueprint' version '3.6.3'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.318+01:00] [INFO ] system-artifacts <DE0004I< Starting plan 'org.eclipse.virgo.kernel.userregion.blueprint' version '3.6.3'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.323+01:00] [INFO ] system-artifacts <DE0004I< Starting bundle 'org.eclipse.gemini.blueprint.core' version '1.0.2.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.325+01:00] [INFO ] system-artifacts <DE0004I< Starting bundle 'org.eclipse.gemini.blueprint.extender' version '1.0.2.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.586+01:00] [INFO ] start-signalling-1 <DE0005I< Started bundle 'org.eclipse.gemini.blueprint.core' version '1.0.2.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.831+01:00] [INFO ] system-artifacts <DE0004I< Starting bundle 'org.eclipse.gemini.blueprint.io' version '1.0.2.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.833+01:00] [INFO ] system-artifacts <DE0004I< Starting bundle 'org.eclipse.virgo.kernel.agent.dm' version '3.6.3.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.837+01:00] [INFO ] start-signalling-2 <DE0005I< Started bundle 'org.eclipse.gemini.blueprint.io' version '1.0.2.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.839+01:00] [INFO ] system-artifacts <DE0004I< Starting bundle 'org.eclipse.virgo.kernel.deployer.dm' version '3.6.3.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.841+01:00] [INFO ] start-signalling-1 <DE0005I< Started bundle 'org.eclipse.gemini.blueprint.extender' version '1.0.2.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.842+01:00] [INFO ] start-signalling-3 <DE0005I< Started bundle 'org.eclipse.virgo.kernel.agent.dm' version '3.6.3.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:06:00 | [2016-03-01T13:06:00.845+01:00] [INFO ] system-artifacts <DE0004I< Starting bundle 'org.eclipse.equinox.ds' version '1.4.0.v20120112-1400'.
INFO | jvm 1 | 2016/03/01 13:06:01 | [2016-03-01T13:06:01.003+01:00] [INFO ] system-artifacts <DE0004I< Starting bundle 'org.eclipse.equinox.util' version '1.0.300.v20111010-1614'.
INFO | jvm 1 | 2016/03/01 13:06:01 | [2016-03-01T13:06:01.009+01:00] [INFO ] start-signalling-3 <DE0005I< Started bundle 'org.eclipse.equinox.ds' version '1.4.0.v20120112-1400'.
INFO | jvm 1 | 2016/03/01 13:06:01 | [2016-03-01T13:06:01.029+01:00] [INFO ] system-artifacts <DE0004I< Starting configuration 'osgi.console' version '0.0.0'.
INFO | jvm 1 | 2016/03/01 13:06:01 | [2016-03-01T13:06:01.035+01:00] [INFO ] start-signalling-3 <DE0005I< Started bundle 'org.eclipse.equinox.util' version '1.0.300.v20111010-1614'.
INFO | jvm 1 | 2016/03/01 13:06:01 | [2016-03-01T13:06:01.038+01:00] [INFO ] system-artifacts <DE0005I< Started configuration 'osgi.console' version '0.0.0'.
INFO | jvm 1 | 2016/03/01 13:06:01 | [2016-03-01T13:06:01.085+01:00] [INFO ] start-signalling-3 <DE0005I< Started bundle 'org.eclipse.virgo.kernel.deployer.dm' version '3.6.3.RELEASE'.
INFO | jvm 1 | 2016/03/01 13:06:01 | [2016-03-01T13:06:01.086+01:00] [INFO ] start-signalling-3 <DE0005I< Started plan 'org.eclipse.virgo.kernel.userregion.blueprint' version '3.6.3'.
INFO | jvm 1 | 2016/03/01 13:06:01 | [2016-03-01T13:06:01.495+01:00] [INFO ] system-artifacts <DE0000I< Installing plan 'org.eclipse.virgo.web.tomcat' version '3.6.3'.
INFO | jvm 1 | 2016/03/01 13:06:01 | [2016-03-01T13:06:01.499+01:00] [INFO ] system-artifacts <DE0000I< Installing configuration 'org.eclipse.virgo.web' version '0.0.0'.
INFO | jvm 1 | 2016/03/01 13:06:01 | [2016-03-01T13:06:01.503+01:00] [INFO ] system-artifacts <DE0000I< Installing bundle 'org.eclipse.virgo.web.spring.integration' version '3.6.3.RELEASE'.
|
Finally it started without any problems. I hope this might help someone else who might help same problem with vSphere Web Client.