Deploy AppMon agents in Amazon EC2

Overview

Amazon EC2 is Amazon’s IaaS cloud offering that lets you to deploy anything from native to java applications. You can learn more on the Amazon Elastic Compute Cloud Website

Adding AppMon to an EC2 image

EC2 uses the concept of images similar to VMWare. The best way to start is to use one of the predefined Windows or Linux Images. Start from an image if you already have one.

Simply copy the Agent MSI or installer .jar to the image and install it. Because the agent boot straps itself, it does not require manual update.

If working with Java, you can configure the agent like you would in any normal installation. However, instead of supplying a hard coded AppMon Server IP, it is automatically a parameter of the image.

Every EC2 instance has meta-data assigned to it. This meta or user data can be used to parametrize an image. Inside the image you would add a script prior to your JVM start:

`#!/bin/sh
# Linux shell script to retrieve the AppMon Server IP from EC2 meta-data
USER_DATA=curl http://169.254.169.254/latest/user-data
if [ -z "$USER_DATA" ]
then
  echo "User Data was not supplied. Do Nothing"
  exit 1
fi

export DT_SERVER=echo $USER_DATA|grep "dynaTraceServer="|sed "s#.*dynaTraceServer=\([^ ]*\).*#\1#g"`

The resulting User Data would be similar to the following:

dynaTraceServer=10.1.1.90

Use the defined environment variable instead of a hardcoded AppMon Server or Collector IP. You can also do this for Windows and .NET, but it is actually easier to have a couple of lines of java code to do this. You can also use Powershell if you are familiar with it. For .NET, make sure that the environment variable is named DT_SERVER. Set it in the same context as the starting CLR. You might also execute the following to automatically change the AppMon .NET Agent Configuration:

REG ADD "HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\dynaTrace\Agent\Whitelist\1"  /f /v server /t  REG_SZ /d %DT_SERVER%

Check the entry with regedit.exe to ensure that you pick the correct Whitelist entry. Then create a new image based on the modification and use it in the future.

AppMon Server in EC2

The AppMon Server setup in EC2 works pretty much like on premise. Use a 64-bit Amazon Linux Image with at least 8 GB memory and sufficient EC Units of CPU (check the deployment guide. You can use Amazon’s Oracle RDS, install your own Oracle or use PostGres as a Performance Warehouse.

Once installed use the AppMon Server init.d script to start the AppMon server on Image startup. The startup script ensures the AppMon Server is running correctly in EC2. It reads the user data to extract the host for the AppMon Performance Warehouse. In your Performance Warehouse configuration dialog box, use dynatracePerformanceWarehouse.localdomain as the hostname.

To use a local PostGres instead, just remove that piece from the startup script and ensure that PostGres is started prior the AppMon Server.

Finally you need to create an Image based on the instance.

AppMon Collector in EC2

Install an AppMon collector if running the AppMon Server on premise or to run a real production environment. See the deployment guide for AppMon collector sizing information. Start the AppMon collector with an init.d script. If you run the AppMon Server in EC2, parameterize it similar to the Agent.

`#!/bin/sh
# Linux shell script to retrieve the AppMon Server IP from EC2 meta-data
USER_DATA=curl http://169.254.169.254/latest/user-data
if [ -z "$USER_DATA" ]
then
  echo "User Data was not supplied. Do Nothing"
  exit 1
fi

export DT_SERVER=echo $USER_DATA|grep "dynaTraceServer="|sed "s#.*dynaTraceServer=\([^ ]*\).*#\1#g"

#start collector
dtcollector --server $DT_SERVER`

The resulting user data is similar to the following:

dynaTraceServer=10.1.1.90

Create an image for the AppMon Collector. You can start multiple Collector instances based on the number of instances that you want to monitor. When starting the instances simply configure the IP of the AppMon Collector in the meta-data instead of the AppMon Server.

The firewall

Amazon EC2 Instances are secured by a firewall that is maintained outside the actual instance. Learn how to use Security Groups in the Amazon Documentation. What is important for a successful AppMon setup is that the incoming ports on the AppMon Server and Collector must be opened in those firewall rules. Otherwise, the Agent cannot connect and you cannot connect your Client to the AppMon Server.

Tying it together

The easiest way to start an environment and tie it all together is using Amazon Cloud Formation. It lets you configure and start multiple instances and wire them together. A sample formation would is similar to the following:

{
  "AWSTemplateFormatVersion" : "2010-09-09",
  "Description" : "easyTravel Demo-in-the-Cloud setup",

  "Parameters" : {
    "KeyName" : {
      "Description" : "Name of and existing EC2 KeyPair to enable SSH access to the instance",
      "Type" : "String"
    }
  },


  "Resources" : {

    "dynaTraceServer" : {
      "Type" : "AWS::EC2::Instance",
      "Properties" : {
	"InstanceType" : "c1.medium" ,
        "SecurityGroups" : [ { "Ref" : "SSH" }, { "Ref" : "HTTP" }, { "Ref" : "dynaTrace" } ],
        "KeyName" : { "Ref" : "KeyName" },
        "ImageId" : "ami-dynaTrace",
	"UserData" : {
                "Fn::Base64" : {
		    "Fn::Join" : [ "\n", [
			    { "Fn::Join" : [ "=", [ "dynaTracePerformanceWarehouse", "localhost" ] ] }
				] ]
                 } },
        "Tags" : [ {
		    "Key" : "Name",
		    "Value" : "dynaTrace Server"
		  }
                 ]
      }
    },


    "MyJavaInstance" : {
      "Type" : "AWS::EC2::Instance",
      "Properties" : {
	"InstanceType" : "t1.micro" ,
        "SecurityGroups" : [ { "Ref" : "SSH" }, { "Ref" : "HTTP" }, { "Ref" : "MyOwnPorts" } ],
        "KeyName" : { "Ref" : "KeyName" },
        "ImageId" : "MyAMI",
	"UserData" : {
                "Fn::Base64" : {
		    "Fn::Join" : [ "\n", [
			    { "Fn::Join" : [ "=", [ "dynaTraceServer", { "Fn::GetAtt" : [ "dynaTraceServer" , "PrivateIp"] } ] ] }
				] ]
                 } },
        "Tags" : [ {
		    "Key" : "Name",
		    "Value" : "My EC2 Instance"
		  }
          ]
      }
    },
    "SSH" : {
      "Type" : "AWS::EC2::SecurityGroup",
      "Properties" : {
        "GroupDescription" : "Enable SSH access via port 22",
        "SecurityGroupIngress" : [ {
          "IpProtocol" : "tcp",
          "FromPort" : "22",
          "ToPort" : "22",
          "CidrIp" : "0.0.0.0/0"
        } ]
      }
    },
    "HTTP" : {
      "Type" : "AWS::EC2::SecurityGroup",
      "Properties" : {
        "GroupDescription" : "Enable http and https",
        "SecurityGroupIngress" : [ {
          "IpProtocol" : "tcp",
          "FromPort" : "80",
          "ToPort" : "80",
          "CidrIp" : "0.0.0.0/0"
        }, {
          "IpProtocol" : "tcp",
          "FromPort" : "443",
          "ToPort" : "443",
          "CidrIp" : "0.0.0.0/0"
        } ]
      }
    },
    "dynaTrace" : {
      "Type" : "AWS::EC2::SecurityGroup",
      "Properties" : {
        "GroupDescription" : "Enable agent, collector and web client port",
        "SecurityGroupIngress" : [ {
          "IpProtocol" : "tcp",
          "FromPort" : "9998",
          "ToPort" : "9998",
          "CidrIp" : "0.0.0.0/0"
        },
	{
          "IpProtocol" : "tcp",
          "FromPort" : "6998",
          "ToPort" : "6998",
          "CidrIp" : "0.0.0.0/0"
        },
        {
          "IpProtocol" : "tcp",
          "FromPort" : "8021",
          "ToPort" : "8021",
          "CidrIp" : "0.0.0.0/0"
        } ]
      }
    },
    "MyOwnPorts" : {
      "Type" : "AWS::EC2::SecurityGroup",
      "Properties" : {
        "GroupDescription" : "My Own Ports",
        "SecurityGroupIngress" : [ {
          "IpProtocol" : "tcp",
          "FromPort" : "1697",
          "ToPort" : "1698",
          "CidrIp" : "0.0.0.0/0"
        },
        {
          "IpProtocol" : "tcp",
          "FromPort" : "1527",
          "ToPort" : "1527",
          "CidrIp" : "0.0.0.0/0"
        } ]
      }
    }
  },

  "Outputs" : {
    "MyInstanceIP" : {
      "Description" : "Public IP of the Web Launcher instance",
      "Value" : { "Fn::GetAtt" : [ "MyJavaInstance", "PublicIp" ] }
    },
    "dynaTraceServer" : {
      "Description" : "Public IP of the dynaTrace Server instance",
      "Value" : { "Fn::GetAtt" : [ "dynaTraceServer", "PublicIp" ] }
    },
    "dynaTraceServerURL" : {
      "Description" : "Web-URL of the dynaTrace Server",
      "Value" : { "Fn::Join" : [ "", ["https://",{ "Fn::GetAtt" : [ "dynaTraceServer", "PublicIp" ] }] ] }
    },
    "dynaTraceWebstart" : {
      "Description" : "URL of Webstart Client",
      "Value" : { "Fn::Join" : [ "", ["https://",{ "Fn::GetAtt" : [ "dynaTraceServer", "PublicIp" ] }, "/webstart/Client/client.jnlp"] ] }
    }
  }
}

Our own instance gets some meta-data assigned. This meta data assigns the internal IP of the AppMon Server. The important thing to notice is that AppMon uses the internal IPs for the agent to collector/server configuration as well as for AppMon Server to Performance Warehouse. Internal communication is free. Using public IPs would incur additional cost and higher latency.

You can also configure the firewall rules here.

Once you start the stack you are good to go.

Next steps

The next logical step is to setup EC2 Cloud Monitoring using the Amazon EC2 Fastpack.