viernes, 24 de octubre de 2014

...make JBDS faster for SwitchYard

While working with SwitchYard, JBoss Developer Studio can be a pain in the ass. Red Hat is working to provide some better user experience, but in the meantime, you can try some of these tips.

Increase heap memory

Eclipse needs a lot of memory, and maven and SwitchYard projects even more, so provide with a good amount of it to your eclipse. Modify jbdevstudio.ini in the appropiate section:
-vmargs
-XX:MaxPermSize=256m
-Xms2G
-Xmx2G
-XX:-UseParallelGC
-XX:+AggressiveOpts
-XX:-UseConcMarkSweepGC
Provide a good amount of memory, so if you can give 3 or 4 GBs instead of 2 better.

Disable automatic updates

Faster startup time. You’ll update or check for updates whenever you want.
Preferences --> Automatic updates --> (Disable) Automatically find new updates and notify me

Disable auto build

If you build whenever you want, the project you want, then disable. If you have a big ammount of projects you can skip having eclipse doing background building all the time. If you have few projects, you can keep it.
Project --> Build automatically (Uncheck)

Refresh workspace on startup

If you don’t do things on command line, then your workspace should be refreshed. If you use git (command line) or maven (command line) maybe you want to keep it:
General -> Startup and shutdown -> Refresh workspace on startup (Enable)

Disable validations

If you there is a lot of background task processing gone on validating your project (due to any facet your project has, like JPA, …​)
Validation -> Suspend all validations (check)

Disable startup features not needed (FUSE, …​)

Use the fewer plugins needed for your work.
General -> Startup & shutdown ->  Plugins activated on Startup (Remove FUSE, Fabric8, JBoss Central, Forge UI, JBoss Tools reporting, Equinox autoupdate)

Disable XML Honour schemas

There is a known bug in JBDS and SwitchYard, so avoid it with:
XML -> XML Files -> Validation -> Honour all XML schema locations (uncheck)

Close JPA and EAR projects if not working on them

Every project that you have opened is both, eating resources and having to be check with the background tasks, so close them if not needed (as dependencies or at all)

viernes, 17 de octubre de 2014

... using vaulted properties in SwitchYard

Description

Usually we define externalize configuration in properties files to just make the application configuration independent. But one problem we face is that this information is usually endpoints and login credentials, like username and password. For this first type of information, there is usually no need to have them encrypted, but there typically is a requirement for this when storing credentials information.
I’m going to explain how to do this in SwitchYard.
Thanks to Nacim Boukhedimi for this post
The steps to be able to use vaulted configuration is:
  • Create a vault file and store the property there
  • Configure the application to read

Create a vault file

Vault mechanism is the mechanism provided by JBoss EAP 6 to enable you to encrypt sensitive strings and store them in an encrypted keystore. This mechanism relies upon tools that are included in all supported Java Development Kit (JDK) implementations. Below is a step by step description of how to encrypt your properties stored in an external properties file.
  1. Create a directory to hold your keystore and other important information. (for instance ${jboss.server.config.dir})
  2. Enter the following command to create a keystore file named vault.keystore:
    keytool -genkey -alias vault -keyalg RSA -keysize 1024 -keystore vault.keystore
  3. Mask the Keystore Password and Initialize the Password Vault. From EAP_HOME/bin folder, run vault.sh and start a new interactive session
    The salt value, together with the iteration count (below), are used to create the hash value.
    Make a note of generated vault Configuration in a secure location. It will be used in the next step to configure EAP to use vault.
  4. Configure JBoss EAP 6 to Use the Password Vault. Copy and paste the generated vault Configuration in EAP config file:
    <vault>
    <vault-option name="KEYSTORE_URL" value="${jboss.server.config.dir}/vault.keystore"/>
    <vault-option name="KEYSTORE_PASSWORD" value="MASK-18JTA2ZfD4eISrndbFgJRk"/>
    <vault-option name="KEYSTORE_ALIAS" value="vault"/>
    <vault-option name="SALT" value="8675309K"/>
    <vault-option name="ITERATION_COUNT" value="50"/>
    <vault-option name="ENC_FILE_DIR" value="${jboss.server.config.dir}/vault/"/>
    </vault><management>

Store your data in the vault

  1. Store encrypted Sensitive strings in the Java Keystore:
    1. run vault.sh and start a new interactive session
    2. Enter the path to the keystore, the keystore password, vault name, salt, and iteration count to perform a handshake.
    3. Select the option 0 to store a value.
    4. Enter the value, vault block, and attribute name.
    5. As a result, the value is stored and a message is displayed showing the vault block, attribute name, shared key, and advice about using the string in your configuration.
      Vault Block:container
      Attribute Name:file.name
      Configuration should be done as follows:
      VAULT::container::file.name::1
      Please make note of the vaultID to use in your properties file

Use the encrypted property in your properties file

Now it is time to use the property. As we want the property to be externalized, we will define the property in a properties file, and then instruct the server to use that properties file.
  1. In your properties, refer to this property as:
 file.name=${VAULT::container::file.name::1}
You need to start the server with the P option to provide the path to the properties file.
./standalone.sh  -P=file:<path_to_properties>

Use the property in your switchyard.xml

You can refer then to this property from your switchyard.xml
      <file:binding.file>
        <file:directory>/tmp/input</file:directory>
        <file:fileName>${file.name}</file:fileName>
        <file:consume>
          <file:initialDelay>10</file:initialDelay>
          <file:delay>10</file:delay>
          <file:delete>true</file:delete>
        </file:consume>
      </file:binding.file>
The file endpoint will be scanning for files under /tmp/input/<your secret filename>

jueves, 16 de octubre de 2014

...how to read a file programmatically in SwitchYard

I’ve been asked recently how to read a file programmatically, based on a cron, or when a request is made via a call to a Service. I didn’t have an answer, well, really I had an answer and I was wrong. Today, somebody brought the topic back, and I revisited the problem, and now I have an answer, one that works, so a good answer.

Problem description

I have the following use case:
A file should be downloaded by FTP from a remote server. The download of the file can be triggered in two ways:
  • by a scheduler whose polling interval may be changed at runtime
  • by a HTTP GET Request on a specified url. The response of the HTTP call must include transformed data based on the downloaded file.
What is a good way to achieve these requirements with SY/Camel? I would like to use the bindings provided by Camel, but have no idea how they can be triggered (e.g. the FTP binding) from outside the binding.

Solution

I have created a sample project, just to demonstrate how it works, not the full problem is solved. You have to use thecontent ennricher EIP.

We have a service, with a scheduled binding (Every 30 secs) that calls a Camel component. In this component, we use pollEnrich to read a file, and the use the contents of the file (the Message) to write into a File (a Reference with file binding).

This is the route we will be using. Note the pollEnrich to read the file.
public void configure() {
 from("switchyard://ProgramaticReload")
 .log("Going to read the file")
 .pollEnrich("file:///input?fileName=in.txt",1000)
 .log("Read from file : ${body}")
 .to("switchyard://FileWriterService");
}
The project is available in github

...SwitchYard in docker

Introduction

Now that there is an official docker image for SwitchYard, I’m going to explain some usages of this image to make you productive with docker and SwitchYard. This way, you’ll be able to have reproducible environments for:
  • development
  • demos
  • support/bug hunting
  • integration / testing
Let’s get us started:

SwitchYard docker official image

The official images, are maintained by Red Hat, and are published to the docker hub. You can browse the source code for the images. Currently there is only support for SwitchYard on Wildfly. Of course, on the official Wildfly image, also supported by Red Hat.
All of the JBoss community projects that have a docker image, are published in the official docker community site. From there, you can link to the individual projects. As I’m going to talk about SwitchYard, let’s see what’s there:

Get started

In order to get you started you just need to download the image
docker pull jboss/switchyard-wildfly
and create a container
docker run -it jboss/switchyard-wildfly
or in domain mode
docker run -it jboss/switchyard-wildfly /opt/jboss/wildfly/bin/domain.sh -b 0.0.0.0 -bmanagement 0.0.0.0
You’ll see that the base SwitchYard docker container is not so useful (doesn’t have an admin user), so the best thing to do, is to extend it.

Extending the SwitchYard container

First thing you need to know is what you want the container for, so let’s see some sample uses:

Development

In this use case, what you want is a container where to deploy your applications, those that you are developing, so it can be useful to have:
  • An admin user created
  • Debug port exposed
  • Binding to all interfaces, so you can access without forwarding (if required)
  • Some volumes to access your information in the container
  • Some volumes (to test file services)
Let’s create a container for this:
Dockerfile
FROM jboss/switchyard-wildfly

#
# Add admin user
RUN $JBOSS_HOME/bin/add-user.sh admin admin123! --silent

#
# Enable port for debugging with --debug
EXPOSE 8787

#
# Volumes
VOLUME /input
VOLUME /output

# Enable binding to all network interfaces and debugging inside the server
RUN echo "JAVA_OPTS=\"\$JAVA_OPTS -Djboss.bind.address=0.0.0.0 -Djboss.bind.address.management=0.0.0.0\"" >> $JBOSS_HOME/bin/standalone.conf

ENTRYPOINT ["/opt/jboss/wildfly/bin/standalone.sh"]
CMD []
I’m going to explain each and every command:
  1. FROM jboss/switchyard-wildfly, this is the base image, so we are reusing everything in there (jboss user, EAP_HOME set, and more. See doc for reference)
  2. RUN $JBOSS_HOME/bin/add-user.sh admin admin123! --silent, this creates and admin user with admin123! as password.
  3. EXPOSE 8787 adds debug port exposure (8787 is the default one in wildfly.
  4. VOLUME /inputVOLUME /output adds two volumes to the image, so we can easily acces or export this volumes if needed. They are very handy when developing file services, as we will see when we run the container.
  5. RUN echo "JAVA_OPTS=\"\$JAVA_OPTS -Djboss.bind.address=0.0.0.0 -Djboss.bind.address.management=0.0.0.0\"" >> $JBOSS_HOME/bin/standalone.conf this adds default binding to any interface (by default only 127.0.0.1) so we can access the server via the ip or name.
  6. ENTRYPOINT ["/opt/jboss/wildfly/bin/standalone.sh"] defines default command to be executed. I like to set the command here as this image will be single purpose, so no need to be able to switch (although it is possible with --entrypoint on command line).
  7. CMD [] this is the arguments to the entrypoint, none by default, but it allows you to set on command line things like "--debug" to enable debugging mode in the container.
Now it is time to build the image:
docker build -t "jmorales/switchyard-dev" .
You can name the image as you like, as this image will be yours. Just need to think that has to have meaningul name so it is easy to remember when used.
Let’s now go to run the image:
docker run -it jmorales/switchyard-dev
This is the simplest command to run the container, but we are not using many of the power we have provided to it, so, let’s make a better command:
docker run -it -name "switchyard" -p 8080:8080 -p 9990:9990 jmorales/switchyard-dev
Now we can access the console with the admin/admin123! user that we have created.
Still not there yet. We need to be able to use file services (remember that in the container the directories will be /input and /output) and we want to debug in the container from the JBDS.
docker run -it -name "switchyard" -v /tmp/input:/input -v /tmp/output:/output -p 8080:8080 -p 9990:9990 -p 8787:8787 jmorales/switchyard-dev --debug
And even maybe use a different profile
docker run -it -name "switchyard" -v /tmp/input:/input -v /tmp/output:/output -p 8080:8080 -p 9990:9990 -p 8787:8787 jmorales/switchyard-dev --debug -c standalone-full.xml
Important things to note here:
  • The image is run in foreground, so we will be viewing the console log. Control-c will stop the container. Since we have the container created, if we want to execute it again, it is just a:
docker start switchyard
  • If you want a disposable container, you can add --rm option to the command line, and every time you stop the container it will be removed. (This option is more useful on other use cases).
  • If you always work with some options, you can just add them to your Dockerfile and create your image with these options, so no need to type them when creating the container (like the profile, --debug, ..)
Of course, this line is very long, and difficult to remember, so we can create a simple alias:
alias docker_switchyard_create="docker run -it -name "switchyard" -v /tmp/input:/input -v /tmp/output:/output -p 8080:8080 -p 9990:9990 -p 8787:8787 jmorales/switchyard-dev --debug -c standalone-full.xml"
alias docker_switchyard_start="docker start switchyard"
Once you are done, you can just delete this container. If you want a fresh instance, or just not working on it anymore:
docker rm -vf switchyard
It is important to delete the volumes if the container has volumes (with the -v) as otherwise this volumes will remain in your docker internal filesystem.

Demos

The demo use case is somehow different as you’ll probably have some application developed that you want to have in acontainer ready to start, so the container for demos will be an extension of the previous one, adding the application/configruation that you want.
FROM jboss/switchyard-wildfly

#
# Add admin user
RUN $JBOSS_HOME/bin/add-user.sh admin admin123! --silent

#
# Volumes
VOLUME /input
VOLUME /output

# Enable binding to all network interfaces and debugging inside the server
RUN echo "JAVA_OPTS=\"\$JAVA_OPTS -Djboss.bind.address=0.0.0.0 -Djboss.bind.address.management=0.0.0.0\"" >> ${JBOSS_HOME}/bin/standalone.conf

ADD myapp.war $JBOSS_HOME/standalone/deployments/
ADD standalone.xml $JBOSS_HOME/standalone/configuration/

ENTRYPOINT ["/opt/jboss/wildfly/bin/standalone.sh"]
CMD []
This example above has 3 main differences with the development one:
  • It removes the exposure of the debug port (not really neccesary)
  • It adds an application to the deployments dir ADD myapp.war $JBOSS_HOME/standalone/deployments/. This app has to be on the same directory as the Dockerfile
  • It adds some configuration to the server ADD standalone.xml $JBOSS_HOME/standalone/configuration/There are many ways to customize the configuration of your wildfly/EAP image, but this is one of the simplest.
And of course, if you had your image from the development use case created, you can just simply extend it.
FROM jmorales/switchyard-dev

#
# Add customizations
ADD myapp.war $JBOSS_HOME/standalone/deployments/
ADD standalone.xml $JBOSS_HOME/standalone/configuration/
Now, time to build your image:
docker build -t "jmorales/myapp-demo" .
And to run it
docker run -it --rm -v /tmp/input:/input -v /tmp/output:/output -p 8080:8080 -p 9990:9990 jmorales/myapp-demo
This time I’ve added --rm to the command line, so when I stop the container, it gets deleted. As this is a demo with everything there, if I need to do it again, just run the container with the same run command line.

Support

The support use case is for somebody to make the environment reproducible, so if sombeody is experiencing a problem, he can create a docker container to isolate the app and the problem, and send it back to the support engineers. Also, some times, we want to test one app with different patches, so can be useful to have all the different versions of the server container as images, so we can try an app in them.
Let’s assume we have to try a feature in SwitchYard 1.1.1 and 2.0.0, and that we have an image for both containers:
  • jboss/switchyard:1.1.1
  • jboss/switchyard:2.0.0
We can just spin up both containers and test the app to see if it works
docker run -it --rm -P jboss/switchyard:1.1.1
docker run -it --rm -P jboss/switchyard:2.0.0
Once we have both containers (just note that if run in foreground, need to run in different terminals, and have different ports exported. I have added -P to auto export ports), we can go to the corresponding consoles and deploy the app and test.
If we want to have our tools set to certain ports (to be predictive) it is easier to have the ports defined in the command line, and use one container at a time

Integration / Testing

In the integration test case, what we want is to have our integration tools spinning up the container for us and deploying the app into this container, do all the integration tests and then stop/destroy the container.
This can easily be achievable with Jenkins (or many of the CI servers out there).
This way we can:
  • Have a clean environment for every test / app
  • Test against many different versions of the Application Server container in an automated way
When adopting testing agains containers, the time for detecting problems lowers so much, that the effort it makes to put it in action pays back very soon.

Tips for developers

As we can develop easily with docker as the container for our Application server, we can easily configure our tools to use this container, so we can have a maven profile to use our containers:
    <profile>
       <id>sy-1</id>
       <properties>
          <jboss-as.hostname>localhost</jboss-as.hostname>
          <jboss-as.port>9999</jboss-as.port>
          <jboss-as.username>admin</jboss-as.username>
          <jboss-as.password>admin123!</jboss-as.password>
       </properties>
    </profile>
    <profile>
       <id>sy-2</id>
       <properties>
          <jboss-as.hostname>localhost</jboss-as.hostname>
          <jboss-as.port>19999</jboss-as.port>
          <jboss-as.username>admin</jboss-as.username>
          <jboss-as.password>admin123!</jboss-as.password>
       </properties>
    </profile>
And we can do a
mvn clean install jboss-as:deploy -P sy-1
to deploy to the SwitchYard 1 container (with port 9999 exported at localhost 9999), or
mvn clean install jboss-as:deploy -P sy-2
to deploy to the SwitchYard 2 container (with port 9999 exported at localhost 19999).