Configuring Tomcat for TLS Authentication Using HSM Devices

To configure ADSS Server Tomcat to use TLS Server Authentication held within a PKCS#11 device (e.g. a PCI-e or network HSM), the following manual configurations must be made:

Note: On 64bit operating systems, PKCS#11 devices cannot contain TLS Server Authentication certificate because this is not supported by Java. Once this is supported by Java, Ascertia will include the latest Java containing support for TLS Server Authentication using PKCS#11 devices.

Open the ADSS Server console and go to the Key Manager module.
Generate a new key pair using the default TLS Server Authentication Template and select a PKCS#11 based crypto profile for key storage. The crypto profile should be set to import the generated certificate to the PKCS#11 device.
On the certificates page generate a new certificate for the TLS Server public key using either (a) the local CA or (b) by generating a PKCS#10 request using the external CA option and then certifying this with an external CA and importing the issued certificate.
Go to the Global Settings > System Certificates page and replace the existing TLS Server Authentication certificate with the one created in step-3.
Now stop the ADSS Server Windows Services (Core, Console and Service) or Unix daemons.
Go to the location: [ADSS Server Home]/service/server/conf/ and take a backup of the server.xml file.
Open the server.xml file in a text editor.
For each of the ports 8778 and 8779, a couple of connector tags are defined. Find the instances of the 8778 and 8779 connector tags which are currently enabled (un-commented) and comment these out of use. Now un-comment the other instances of the 8778 and 8779 connector tags which are disabled (commented) so that these will be used.
In the un-commented connector tag i.e. 8778 find the attribute keyAlias and provide the key alias for the TLS Server Authentication certificate as shown on the PKCS#11 vendor software. Follow these instructions to get the relevant key alias from the PKCS#11 device:

Go to the location: [ADSS Server Home]/util/bin and copy this path
Launch the command prompt and change directory to the location mentioned above
Execute the command pkcs11_ssl_key.bat or simply double click the pkcs11_ssl_key.bat file
This displays the keyAlias for the configured TLS Server Authentication key as shown below:

Now also add the same keyAlias to the un-commented connector tag 8779
For both of the connectors i.e. 8778 and 8779 modify the value for the element keystorePass and set it to the PKCS#11 device PIN and save server.xml file
Repeat steps 6 to 9 and 11 for the server.xml file present at the location: [ADSS Server Home]/console/server/conf/. Note that the TLS port in the console/server/conf/server.xml file is 8774. The key alias for the TLS Server authentication certificate should be configured for this port only. Also it is not required to run the utility again as you already know the key alias from the HSM device.
Go to the location: [ADSS Server Home]/jdk/jre/lib/security/ and open the file java.security in a text editor and find the property
#security.provider.10=sun.security.pkcs11.SunPKCS11 [ADSS Server Home]/conf/pkcs11.properties
This line should be un-commented and the element shown as [ADSS Server Home] should be replaced with the absolute path for the ADSS Server installation directory and save this file.
Start the ADSS Server Windows Services (or UNIX daemons).