This error is caused by the certificate used by the secure https site not being included in the JBoss Java keystore. See this article S140385 for a tool you can run on the JBoss server to see if the certificate is trusted . See this article S139798 for more on how to import the appropriate certificates into the Java keystore.
SBM has several Java-based components that can make calls to outside services that are protected with SSL. A few of these services are the Notification Server, Mail Client, Orchestration Engine and Application Repository. Orchestrations that use HTTPS SSL endpoints do so under the purview of the Orchestration Engine. When these services attempt to contact an SSL enabled endpoint ( web service , Exchange Server, etc), it is required to tell the SBM JDK to trust the SSL certificate provide to it by the endpoint.
Enter your AR credentials in Composer Options Composer makes a call to the AR web services at http://hostname:port/mashupmgr/services/AppRepositoryService to get the STS URL. If Composer finds the repository, it will return the STS url in this form: http(s)://hostname:8085(8243)/idp/services/ Trust . Using the STS URL, Composer presents it credentials to the STS which authenticates it and returns a token to be included in further communication with the Application Repository.
The reason the error occurs is because the web service wsdl that gets created when SBM is installed references localhost in the wsdl itself instead of the computers host name. You can either fix the problem by editing the wsdl code as it sits in your development area or editing the .wsdl files in the SBM install directory and re-importing the wsdl to your development area.