The Technology Hub: April 2015

Thursday, April 30, 2015

Steps to Task List and Task Group in Forum Sentry. (79/285 technotes for 2015).

Login
Resources –> Security Policies –> SSL

Login
Resources –> PKI –> Keys

IBM BPM Snapshots. (75/285 technotes for 2015)

Snapshots are

Immutable version of Process Application, Toolkit

Key Concepts for IBM BPM 7.5? (74/285 technotes for 2015)

Process Application -

is a packaging and deployment construct
container - which has all the models, data definitions, service models, rules and supporting files to edit and run
can contain 1 or more BPD
can have dependencies on Toolkits
acronym must be unique

Toolkit -

is a way to manage reusable assets
is a collection of BPM library items that can be shared across multiple projects in IBM Process Designer
contain - all of the same Library Items as Process Application
process Application which has dependency with Toolkit - can use all the elements of the toolkit
can only be edited - by users with the write/admin privileges. It is available as Read-Only in the Process Application
Process Application can upgrade to the newer version of the toolkit and revert back very easily.

Snapshots & Tracks - are part of the version management infrastructure.

Participant Groups - are how you manage who can work with different parts of the business

Security Groups - are how you manage who can create/edit and deploy Business Processes during deployment.

Scripting/Programming Language - Javascript is used to define logic and manipulate data

Advance Editor - complex Javascript

Programming Model in IBM BPM - Clear seperation

Model or the Flow
Implementation

Wednesday, April 29, 2015

IBM BPM Design Patterns for Performance. (73/285 technotes for 2015)

BPM - performance

Use high performance disk subsystems
Use most current BPManager and BPMonitor reelase
Appropriate JVM Heap Size for optimal throughput and response time
Tune your DB for optimal performance
Disable tracing and logging
Configure thread pools to ensure sufficient concerrency
Use fast, high bandwidth network connections
For Business Processes - that use

BPMN -

tune the bpd-queue-capacity & max thread pool size - optimal throuput and scalling
remove unnecessary snapshots from Process Center
disable notifications is they are not required
define auth alias, when using CEI to emit events
tune cache parameters
tune participant groups
optimize data search operations
tune Event Manager - tune BPD pool size, and worker thread pool size.
tune number of timer events
tune Process Server DB - increase log file size & enable file system caching

BPEL -

maximize the use of micro flow
use non-incorruptible processes - if macroflow is needed - separate the solution to use microflows and macroflows
For task and process list queries - use composite query tables
Use Work Manager based navigation to improve the throughput for long running processes.
avoid using asynchronous processes unnecessarily.
avoid overly granular transaction boundaries in SCA

Model & Development:

BPEL / Integration Designer:

use short running processes - run in one J2EE Transaction with no persistence – Where possible use synchronous interaction for non-interruptible processes
if macro flow is required create a seperate BPEL
Avoid using - async calls
fewer transactional boundaries - best case use

Join Transaction (Interface of the called component) - true,
Suspend Transaction (Reference of the calling transaction) - false,

Transaction - any global

Services that are automated or start other services - web services, enterprise resource applications

Events:

start and end - 2 events are sufficient

Architecture:

Choose query tables for task list and process list queries.
Choose efficient metadata management -

performance is better if BO types follow the Java naming standards
reuse dataobject type metadata where possible - rather than using DataFactory.create(String URI, String typeName) use DataFactory.create(Type type) -
Business State Machine uses more power … so choose BPEL is possible.

Use - share by reference libraries where possible
Ensure content in toolkit is used in several applications.
BO parsing mode - Applications which Data Streams - like Webservices, SCA components benefit from lazy parsing.

BPMN / BPD:

Involve Human Interactions
Business Process - should have a Business Significance … and not just programming - level granularity

Events:

use of Events (CEI) - should be for Business Monitoring - as event emission uses persistent mechanism
Business Monitoring should not be confused with IT Monitoring
Clients are concerned about state of their business - so use events to track when a task completes / or changes state

Architecture:

Fast connection between Process Designer and Process Center
Minimize the use of Service Tasks -
Use searchable business variables - with caution
minimize the number of variables used
minimize the size of each variable used
set variables to null when no longer use - like DB result sets
turn off auto-tracking - auto tracking in BPD is not required - enabled by default - if required create tracking groups to track key events.
avoid BPDs that run perpetually / continuously – as it polls continuously for events - alternatives - use JMS service or UCA
develop efficient coaches

Avoid large complex coaches
Avoid large / repeating tables - use pagination
Always wire coaches to end node

Minimize use of Large Javascript scripts - it might produce large DOM models - which is expensive - limit it to 50 lines
Avoid direct sql access to internal BPM tables - such as LSW_TASK or LSW_PROCESS - use BPM javascript / rest apis

Development Best Practice:

clear variables in exposed human services that are not intended to end
donot use multi instance loops in the system lane or for batch activities
use conditional joins only when necessary
Avoid global error handling
Use sequential System lane activities efficiently.
Ensure Process Center is tuned
preventing wsdl validation from slow web service performance

References:

http://www.evernote.com/l/AKZrhhusWANG75yXr6HJbHEDMoWmyCvplqM/

What is ENI (Elastic Network Interface) in AWS? (72/285 technotes for 2015)

Elastic Network Interface is a Virtual Network Interface that you can attach to an instance in VPC.

NIC - A network interface is the point of interconnection between a computer and a private or public network. A network interface is generally anetwork interface card (NIC), but does not have to have a physical form. Instead, the network interface can be implemented in software.

For eg: Loop back address or 127.0.0.1 is not a physical device - but a software simulating a network device

References:

http://bit.ly/1bEtFc0

What is Reverse DNS for email applications? (71/285 technotes for 2015)

If you plan to send email from one of the instances

Provision and EIP for the instance.
Request to remove email sending limitations with the EIPs.
AWS works with ISPs and Internet Spamming Organization - Spamhaus to reduce the email sent be flagged as spam

When should we use instances with multiple IP addresses? (70/285 technotes for 2015)

It can be useful to have multiple IP addresses to your instance in VPC to do the following

Host multiple websites on a single server by using multiple SSL Certificates on a single server and associating each server with a specific ip address.
Operate network appliances, such as firewalls and load balancers that have multiple private ip addresses for each network interface
Redirect internal traffic to a standby inatance in case your instance fails by re-assigning the secondary private ip address to the standby instance

References:

http://amzn.to/1zq6M7g

What is a Network ACL? (69/285 technotes for 2015)

A network ACL is an optional layer of security that acts as a firewall for controlling traffic in and out of a subnet.

What is an Amazon VPC? (68/285 technotes for 2015)

Amazon VPC (Virtual Private Cloud) is a

virtual network
Logically isolated area in the AWS cloud

Benefits: By launching your instances in VPC

Assign static IP addresses
Assign multiple IP addresses
Define Network Interfaces and attach multiple Network Interfaces
Change Security Group membership - while running
Control Outbound (egress) / Inbound (ingress) traffic from your instances
Add an additional layer of access control to your instance in the form of Network Access Control Lists (ACL)
Run your instance in a single-tenant hardware.
ELP is associated even when the instance is stopped
Assign private IP address from a range of private IP addresses

C1, C2, C3, C4 are EC2-Classic

C1, C2 - was launched by an account
C3, C4 - was alunched by a different account
Can communicate with each other
Access internet and other services such as S3 directly

V1, V2 are instances

V1 & v2 belong to different subnets in the same VPC
They were launched by the account that owns the VPC
No other accounts can launch instance in this VPC.
V1 & v2 can communicate with each other
V1 & v2 can communicate with C1-C4, S2 and Internet through the Internet Gateway

References:

http://amzn.to/1zpcw0Y

What is an IAM and IAM Role in AWS? (67/285 technotes for 2015)

IAM

IAM (Identity and Access Management) is used to manage access to Compute, Storage, Database and Application Services in the AWS cloud.

Applications can sign their API requests with AWS Credentials.
Securely distribute your AWS credentials to the instances
Applications on those instances to sign requests - while protecting them from otehr users

Challenge - to securely distribute credentials across all instances - especially those created by AWS on your behalf - Spot Instances and Auto Scalling group.

IAM Roles

With IAM Roles applications can securely access AWS service API from EC2 instances.
Now you can create IAM role, which has a set of permissions, and launch EC2 instances with the IAM role.

References:

https://www.youtube.com/watch?v=Ul6FW4UANGc

http://amzn.to/1zp4Cow

Trouble shooting working with roles

What is IAAS, PAAS and SAAS with examples? (66/285 technotes for 2015)

* This is a very good discussion found in StackOevrflow - with little modification for my notes - you can find the actual discussion here*

IaaS, PaaS and SaaS are cloud computing service models

IaaS (Infrastructure as a Service), as the name suggests, provides you the computing infrastructure, physical or (quite often) virtual machines and other resources like virtual-machine disk image library, block and file-based storage, firewalls, load balancers, IP addresses, virtual local area networks etc.

Amazon EC2, S3, EBS, ELB, VPC, Route53
Windows Azure,
Rackspace,
Google Compute Engine.

PaaS (Platform as a Service) - , as the name suggests, provides you computing platforms which typically includes operating system, programming language execution environment, database, web server etc.

Aamzon -

RDS, Elastic Beanstalk, Dynamo DB, RedShift
Windows Azure,
Heroku,
Force.com,
Google App Engine,
Apache Stratos.

While in SaaS (Software as a Service) model you are provided with access to application softwares often referred to as on-demand softwares. You don’t have to worry about the installation, setup and running of the application. Service provider will do that for you. You just have to pay and use it through some client.

Google Apps,
Microsoft Office 365.

Few additional points regarding your question:

1- AWS(Amazon web services) is a complete suite which involves a whole bunch of useful web services. Most popular are EC2 and S3 and they belong to IaaS service model.

2- Although Hadoop is based on previous works by Google(GFS and MapReduce), it is not from Google. It is an Apache project. You can find more here. It is just a distributed computing platform and does not fall into any of these service models, IMHO.

3- Microsoft’s Windows Azure is again an example of IaaS.

As far as popularity of these services is concerned, they all are popular. It’s just that which one fits into your requirements better. For example, if you want to have a Hadoop cluster on which you would run MapReduce jobs, you will find EC2 a perfect fit, which is IaaS. On the other hand if you have some application, written in some language, and you want to deploy it over the cloud, you would choose something like Heroku, which is an example of PaaS.

References:

StackOverflow - http://bit.ly/1AgaDP3

Tuesday, April 28, 2015

HTTP Authentication (Basic, Digest). (65/285 technotes for 2015)

HTTP Basic Auth:

Client sends username / password in clear text
should be used only with HTTPS (SSL)

HTTP Digest:

Client sends hashed form of the password to the server
should be used over HTTP (SSL0

Caution: Although the password cannot be captured, it can be replayed.

HTTP Basic Auth - details:

If a server gets a anonymous request for a protected resource, it can immediately reject the request with a 401 (access denied) code, and setting the WWW-authenticate response header as shown below.
Web browser will display a login dialog when this response is received.
User / password is entered
browser sedn all the info back to the server

GET /securefiles/ HTTP/1.1
Host: www.httpwatch.com
Authorization: Basic aHR0cHdhdGNoOmY=

The authorization value “

aHR0cHdhdGNoOmY=” is simply the base 64 encoded version of username:password

What is a default Security Group in AWS? (64/285 technotes for 2015)

If an instance is created without specifying a Security Group, a default Security Group for the VPC will be automatically associated to the instance
Rules

Specifies itself as a source Security Group, This is what allows Inbound traffic only from other instances associated with the default Security Group.
Allow all outbound traffic from this instance.

You cant delete a default security group.

Security Groups and Security Rules in AWS. (63 of 285 technotes for 2015)

Security Group - acts as a virtual firewall that controls traffic for one or more instances.
You can add multiple Security Groups to a instance.
You can add multiple Rules to a Security Group.
The rules from different security groups and aggregated and applied to the instance.

Caution:

Because you can assign multiple rules to a security group, an instance can have hundreds of rules that apply. This can cause complexity.

Suggestion:

Condense your rules as much as possible,

Configure WSDL Libray and WSDL Policy in Forum Sentry. (62 of 285 technotes for 2015)

STEP 1: Create a Library (if does not exist) and Add a WSDL to the Library.

Login to the console - https://localhost:5050/login.do
Click on Gateway –> WSDLPolicies –> WSDL Libraries

New WSDL Library

After adding the Library –> Add WSDL to the Library

Save the Library

STEP2: Add a new WSDL Policy.

Click on Gateway -> WSDL Policy –> New
Complete the details –> Next

Select the operation for WSDL Policy from the Library

Click Next, Next and Done.
Goto Gateway –> WSDL Policy –> Click on the newly created Policy
Click on Settings

Scroll down to Processing Settings

select the settings that is applicable, In this case we want to add UserName Token to the request.
Go back to the WSDL and click on the Port

Scroll down all the way and mark Enable WSDL Access as true.

Monday, April 27, 2015

How to log outgoing/incoming SOAP Requests in Websphere? (61 of 285 technotes for 2015)

Trace for SOAP WS messages can be enabled by following these steps on Websphere 7 administration console:

Go to administration console
Go to: Servers -> WebSphere application servers -> server > Change Log Detail Levels
In the package tree find the package com.ibm.ws.websvcs.trace, click on package name and choose ‘All Mesasges and Traces’, as result you should have log details set as below:

*=info: com.ibm.ws.websvcs.trace.*=all

Click on 'OK’ and then Save the changes.

SOAP request/responses can now be found in trace.log on your appl. server.

References: http://bit.ly/1J4dSy5

Pretty printing SOAP messages. (60 of 285 technotes for 2015)

Handy Java code to print SOAPMessages

References: - http://bit.ly/1dkVWoW

public static String getSOAPMessageAsString(SOAPMessage soapMessage) {
      try {

         TransformerFactory tff = TransformerFactory.newInstance();
         Transformer tf = tff.newTransformer();

         // Set formatting
        
         tf.setOutputProperty(OutputKeys.INDENT, "yes");
         tf.setOutputProperty("{http://xml.apache.org/xslt}indent-amount",
               "2");
         
         Source sc = soapMessage.getSOAPPart().getContent();

         ByteArrayOutputStream streamOut = new ByteArrayOutputStream();
         StreamResult result = new StreamResult(streamOut);
         tf.transform(sc, result);

         String strMessage = streamOut.toString();
         return strMessage;
      } catch (Exception e) {
         System.out.println("Exception in getSOAPMessageAsString "
               + e.getMessage());
         return null;
      }

   }

Saturday, April 25, 2015

Cannot ping a Elastic IP Address in AWS. (59 of 285 technotes for 2015)

Check the Security Group associated to the instance. Most likely the incoming rule for ICMP Echo Request is disabled.

Friday, April 24, 2015

What is Cloud Watch in AWS? (58 of 285 technotes for 2015)

Cloud Watch - collects, processes raw data from EC2, into readable, near realtime metrics.

The Analytics of CloudWatch is very limited. Librato is good option to add value to the data is generated from CloudWatch.

What is DNS? (57 of 295 technotes for 2015)

Domain Name Servers (DNS) are the Internet’s equivalent of a phone book. They maintain a directory of domain names and translate them to Internet Protocol (IP) addresses. This is necessary because, although domain names are easy for people to remember, computers or machines, access websites based on IP addresses.

References:

http://bit.ly/1zUXdrv

What is DHCP? (56 of 285 technotes for 2015)

Dynamic Host Configuration Protocol (DHCP) is a network protocol that enables a server to automatically assign an IP address to a computer from a defined range of numbers (i.e., a scope) configured for a given network. DHCP assigns an IP address when a system is started, for example:Oct 17, 2014

References:

https://kb.iu.edu/d/adov

Thursday, April 23, 2015

Generate Key Pair in AWS? (55 of 285 technotes for 2015)

What is a NAT Instance? (54 of 285 technotes for 2015)

Instances that you launch into a private subnet in a virtual private cloud (VPC) can’t communicate with the Internet.

You can optionally use a network address translation (NAT) instance in a public subnet in your VPC to enable instances in the private subnet to initiate outbound traffic to the Internet, but prevent the instances from receiving inbound traffic initiated by someone on the Internet.

References:

http://amzn.to/1aW2pVE

What is the Difference between EBS and S3? (53 of 285 technotes for 2015)

References:

What is the difference bettwen Elastic IP and Public IP? (52 of 285 technotes for 2015)

Elastic IP address - designed for dynamic cloud computing

static IP address
assigned to your account, until you release it.
there is a small hourly charge for EIP which is not associated to and instance, or if it is associated to an instance, but the instance is turned down.
EIP is limited to 5 per account
IPv4 static IP’s are very scare, so the limitation
EIP should be used only incase of Load Balancing and use DNS hostnames for all other internode communication
You can mask the failure of an instance by rapidly re-mapping the address to another instance of the account.

Public IP Address -

dynamically assigned IP Addresses
also given an external dns hostname
when launching an instance in VPC … public IP is determined by subnet setting
when an instance is stopped – public ip address is released to the Amazon’s pool of public addresses
public ip address is released when it is associated with EIP
if an instance is assigned an EIP, it automatically received a DNS hostname, if DNS hostname is enabled.

Best Practice EIP (Static IP)

Internet IP Address is a scare public resource.
Use EIP for Load Balancers use case.
Use DNS hostnames for all internode communication.

References:

http://amzn.to/1aW1WCS

What is an AMI?(51 of 285 technotes for 2015)

Amazon Machine Image. Its a template that contains software configurations. From an AMI you can launch an Instance which is a virtual server running on the host computer in Amazon’s data center.

What is a Key Pair? (50 of 285 technotes for 2015)

AWS uses public key cryptography to secure login info. A Linux instance has no password. U use the key pair to login to the system. Steps:

Mention the name of the key pair
Provide private key when u login to the system

What is AWS Direct Connect? (49 of 285 technotes for 2015)

Establishes a dedicated network connection from your premises to AWS.

References:

http://aws.amazon.com/directconnect/

How is a private/public IP Address allocated to an Ec2 instance (48/285 technotes for 2015)

Private IP Address:

Launch an instance - private IP address is allocated using DHCP
Private IP address is not reachable from internet
Ecah instance has a default Network Interface - which specifies primary private IP address for the instance
You can specify additional priavate IP address
secondary IP address can be reassigned from one instance to another
each instance is provided an internal DNS name
DNS name is not resolvable from outside the network

Public IP Address:

Is assigned from a pool of Public IP address.
once de-associated, cannot be reused for the same instance
cannot manually associate/de-associate public ip address

when instance is stopped or terminated
instance associated with Elastic IP address

If you want the same IP address for the instance, use a Elastic IP address.

What is the AWS Command Line Interface? (47 of 285 technotes for 2015)

The AWS Command Line Interface is a unified tool to manage your AWS services. With just one tool to download and configure, you can control multiple AWS services from the command line and automate them through scripts.

Download CLI Tools - unzip it and keep it at a location - say c:aws_cliec2-api-tools-1.7.3.2. This is your EC2_HOME.
Set Java_HOME & EC2_HOME environment variables
add %JAVA_HOME%/bin; %EC2_HOME%/bin to the PATH variable
Set

AWS_ACCESS_KEY &

AWS_SECRET_KEY environment variable
execute the command in command promt- c:> ec2-describe-regions (see image below)

All the commands are available at the EC2_HOMEbin directory

You can simple use the MSI (for WIndows) - see below

and set the “Access Key” and the “Secret Key” using the command “aws configure”.

References:

What is the AWS CLI - http://amzn.to/1I0tNzf
AWS CLI - MSI - http://amzn.to/1zUUwGq
Setting up for Linux and Mac - http://amzn.to/1I0tPXO
Setting up for WIndows - http://amzn.to/1I0tZOM
Usage - http://amzn.to/1zUTU3P

The "Big Three" are all in on big data / predictive modeling

kevinmccauley:

Here’s a quick summary on the technology behind next big thing – machine learning

All three major public cloud providers have connected their core storage services, and one or more of their database products, to the machine learning offerings:

Amazon Machine Learning connects to S3, Redshift and the MySQL flavor of its Relational Database Service (RDS).

Google Prediction API can read data from…

View On WordPress

Building Reactive Apps with AWS Lambda

Learn how to use AWS Lambda to react immediately to events from different AWS tools and your own applications.

Building Reactive Apps with AWS Lambda

Preserving Amazon EBS volumes on Instance Termination? (46 of 285 technotes for 2015)

While creating an Instance, at step 4: Add Storage, uncheck Delete on Termination, to preserve Amazon EBS volumes on Instance Termination.

By default we might want to just keep it as is.

Importing & Exporting Instances in AWS. (45 of 285 technotes for 2015)

You can use the AWS VM export Import tool to import VM images from local environment to AWS and convert them to ready to use EC2 instances. Later you can export back to the local environment

Advantages:

Leverage existing investment in building the local instances - configurations, compliance, etc.

Limitations Exporting:

Cannot export

Amazon EBS (Elastic Blcok Storage) data volumes
An instance that has more than virtual disk
Unless it has been previously imported into AWS.

Limitations Importing:

Can have upto 5 import tasks at the same time per region
Task timeout is 7 days
Typically you use a compressed version of the dusk image (<1 TB)
Make sure your VM uses a single disk
Atleast 250 MB of space is required for installing drivers and other s/w
Multiple NIC not supported. After importing your instance will have single NIC with DHCP for address assignment.
IPv6 not suported

References:

http://amzn.to/1DgMjMS

Pelican Static Site Generator, Powered by Python

Completely static site generator - that requires no backend.

The blog http://alexwlchan.net/ is created using this.

Pelican Static Site Generator, Powered by Python

Find Untagged Tumblr Posts

A way to find untagged posts from your Tumblr site

Find Untagged Tumblr Posts

What is a “Single Segment Network”? (44 of 285 technotes for 2015)

A single segment network is a series of logically connected nodes (computers, printers, and so on) operating in the same portion of the network. In a single segment network, Filtering Service and Network Agent must be positioned to monitor Internet traffic across the entire network.

References:

http://bit.ly/1DTCVmh

What is the use of the ip address 169.254.169.254 used in AWS? (43 of 285 technotes for 2015)

These are dynamically configured link-local addresses. They are only valid on a single network segment and are not to be routed.

Of particular note, 169.254.169.254 is used in Amazon EC2 and other cloud computing platforms to distribute metadata to cloud instances.

References:

What is Windows Powershell? (42 of 285 technotes for 2015)

Shell developed by Microsoft for the purpose of Task Automation and Configuration Management
First released in 2006
Based on .Net Framwork
Includes CLI (Command Line Interface) and scripting language
Windows Powershell ISE (Integrated Scripting Environment) - it is a Graphical User Interface (GUI)

Things that you can do with Power Shell

Display all the USB devices
Set time consuming task in the background
Identify and kill processes
Filter specific info about Computers in the network and export in html
Simplify and Automate tedious tasks, repetitive tasks
Powershell is very helpful for Network Admins - working with Active Directory
It has hundreds of customizable commands called “cmdlets”.

References:

http://www.7tutorials.com/simple-questions-what-powershell-what-can-you-do-it

SYNCbits » SYNCing all the little bits of data

Connect your apps!

SYNCbits » SYNCing all the little bits of data

Wednesday, April 22, 2015

What is a SOAPHandler in WS-Security? (41 of 285 technotes for 2015)

SOAP handler is a SOAP message interceptor, which is able to intercept incoming or outgoing SOAP message and manipulate its values.

“javax.xml.ws.handler.soap.SOAPHandler” is the Interface which needs to be implemented by someone who wants to use the SOAP Handler.

It inherits 4 methods namely

getHeaders()
close()
handleFault()
handleMessage()

Any implementation of SOAPHandler interface will implement the handleMessage() operation to intercept the SOAP message.

SampleCode:

public boolean handleMessage(SOAPMessageContext context) {
Boolean isOutboundMessage = (Boolean) context.get(MessageContext.MESSAGE_OUTBOUND_PROPERTY);
if (isOutboundMessage) {
SOAPPart messageSoapPart = context.getMessage().getSOAPPart();
WSSecHeader securityHeader = new WSSecHeader();
try {
securityHeader.insertSecurityHeader(messageSoapPart);
} catch (WSSecurityException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
Map map = new HashMap();
map.put(Constants.MAPPING_ALIAS, “CT_J2C_FDSH_ALIAS”);
CallbackHandler callbackHandler = null;
LoginContext loginContext = null;
try {
callbackHandler = WSMappingCallbackHandlerFactory.getInstance().getCallbackHandler(map, null);
loginContext = new LoginContext(“DefaultPrincipalMapping”, callbackHandler);
loginContext.login();
Subject subject = loginContext.getSubject();
Set credentials = subject.getPrivateCredentials();
PasswordCredential passwordCredential = (PasswordCredential) credentials.iterator().next();
String userid = passwordCredential.getUserName();
String password = new String(passwordCredential.getPassword());
WSSecUsernameToken usernameToken = new WSSecUsernameToken();
usernameToken.setPasswordType(WSConstants.PASSWORD_DIGEST);
usernameToken.setUserInfo(userid, password);
WSSecTimestamp timestamp = new WSSecTimestamp();
usernameToken.build(messageSoapPart, securityHeader);
timestamp.build(messageSoapPart, securityHeader);
} catch (Exception e) {
System.out.println(“FDSHJavaHandler - Exception occured while loggin in”);
}
}
return true;

What is instance metadata and user data in AWS? (40 of 285 technotesfor 2015)

Instance Meta Data:

data about your instance that you can use to configure or manage the running instance.
Instance metadata is divided into categories.

User Data:

user data that you supplied when launching your instance.
you can specify parameters for configuring your instance, or attach a simple script.
You can also use this data to build more generic AMIs that can be modified by configuration files supplied at launch time.

For example, if you run web servers for various small businesses, they can all use the same AMI and retrieve their content from the Amazon S3 bucket you specify in the user data at launch. To add a new customer at any time, simply create a bucket for the customer, add their content, and launch your AMI. If you launch more than one instance at the same time, the user data is available to all instances in that reservation.

Security:

the data is not protected by cryptographic methods. Anyone who can access the instance can view its metadata.
you should take suitable precautions to protect sensitive data (such as long-lived encryption keys).
You should not store sensitive data, such as passwords, as user data.

In AWS use the blow url to call the rest service to get the instance details

http://169.254.169.254/latest/meta-data/

Or
you can use Windows Power Shell

invoke-restmethod -uri http://169.254.169.254/latest/meta-data

or you can download Instance MetaData Query Tool from

http://aws.amazon.com/code/1825

or Linux / Unix

curl http://169.254.169.254/latest/meta-data/

User Data -
Use the below url to retrieve User Meta Data

http://169.254.169.254/latest/user-data

Dynamic Data -
Use the below url to retrieve Dynamic Data

http://169.254.169.254/latest/dynamic

Resources:
http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-instance-metadata.html

Tuesday, April 21, 2015

How to Integrate a third party app to check my AWS billing report? (39 of 285 technotes for 2015)

AWS generates a billing report, which it stores periodically (daily or more) in the dedicated S3 bucket that you have created.

A third party application (like Cloudability) can fetch the billing data and provide analytic on the same. Please follow the link in the reference to see the detailed steps.

The highlights are mentioned below

Enable AWS Billing Report
Creating an S3 Bucket
Generating access credentials

References:

Integrating Cloudability with AMAZON - http://bit.ly/1G3HDfy

AWS Policy Generator

Fine grained Access Control mechanism - The AWS Policy Generator is a tool that enables you to create policies that control access to Amazon Web Services (AWS) products and resources.

Generate Policy for

S3 Bucket Policy
SNS Topic Policy
IAM Policy
SQS Queue Policy

You can copy the policy, paste it into a text file, and then use it with the APIs calls or command-line tools as appropriate.

AWS Policy Generator

What is a Reserved Instances in AWS? (38 of 285 technotes for 2015)

Offering

All upfornt - Pay for a instance ahead for 1-2-3 years.(best cost savings)
Partial upfront - Pay full for partial term, and the remainder term as monthly installments.
No upfront - monthly installment throughout the term. (low savings)
Light utilization -
Medium utilization -

No matter which plan you choose - you are committing to pay for 1-2-3 years.
Benefits

Get significant reduction in cost
Capacity reservation for number of EC2 instances

Modifying Reserved Instances - as long as requirements for modifications exist, and capacities are available.
Unused Reserved instances - can be sold in the Reserved instance market place - https://portal.aws.amazon.com/ec2/ri/seller_registration

References:

TechCrunch - http://tcrn.ch/1G3BH6b
Cloudability - http://bit.ly/1G3BPm8

Monday, April 20, 2015

Define REST. RESTful vs SOAP webservices (37 of 285 technotes for 2015)

DEFINE:

REST stands for REpresentational State Transfer
Architectural style - HTTP is the protocol that is used

like SOAP over HTTP - REST over HTTP

Fundamentals

everything is a resource
every resource is identified by an url
only 4 operations - POST, GET, PUT, DELETE (like CRUD)
Stateless - request / response should contain the context
Representations doen by XML / JSON

COMPARE:

Restful Webservice more popular to SOAP Webservice due to its simplicity - Unless u have a definitive reason to use SOAP, use REST

SOAP vs REST

REST operates CURD operation on the data - getUser(User);
SOAP has operations which have application business logic - switchCategory(User, OldCategory, NewCategory)

REST - Advantages

multiple formats - XML, JSON
JSON is easy and faster to parse
better support for browser clients
reads can be cached - better for performance and scalability

SOAP - Advantages

WS Security and not just SSL (See: SSL Limitations)
WS Atomic Transactions - ACID property
WS Reliable Messaging - with automatic retry facility

SOAP is useful in scenarios where I want to make use of the above features. for eg: Iphone app interacting with bank, where I want to make sure it is complete. Retrying might be catastrophic
In all other cases we can/should use REST.

Reference:

How to convert a java object to json? (36 of 285 technotes for 2015)

Use Jackson - http://jackson.codehaus.org/
Sample code -

ObjectWriter ow = new ObjectMapper().writer().withDefaultPrettyPrinter();
String json = ow.writeValueAsString(wh);

What is AWS Simple Notification Service (SNS)? (35 of 285 technotes for2015)

Amazon Simple Notification Service (Amazon SNS) is just like a JMS Topic

web service that
coordinates and manages the delivery or sending of messages to subscribing endpoints or clients.
In Amazon SNS, there are two types of clients—

publishers and
subscribers—also referred to as producers and consumers.

Push and fan out messages to multiple recievers

Reefrences:

http://docs.aws.amazon.com/sns/latest/dg/welcome.html

What is Auto Scalling Group in AWS? (34 of 285 technotes in 2015)

Auto scaling in AWS is a way to increase/decrease the infrastructure required to run a server in AWS purely based on the demand.

Its helps to scale up/down the EC2 capacity based on the conditions defined.
Automatically increases the number of EC2 instances to handle demand.
Decreases/stops the number of EC2 instances when demand reduces to reduce cost.
Note: all server / nodes should be in auto start mode for Auto scaling to work properly.

For example:

In a typical Websphere Clustered environment with multiple nodes and a Load balancer, we can create a Auto Scaling Group to start /stop nodes based on demand.

References: http://aws.amazon.com/autoscaling/details/

Multi Tenancy vs Dedicated Tenancy Hardware in AWS? (33 of 285tech-notes for 2015)

Does your Business need a shared or private Cloud?

Shared/Multi Tenant
Pro: Price
Cons: security

Advantages:

The largest advantage here is price - popular among small business and individuals
maximum efficiency of computing resources.
vendor can make overnight improvements, like adding features.
Think Gmail and other services which Individuals user
Think Rally, Salesforce which businesses use.

Disadvantages:

who are not bound by compliance standards
less concerned about security.
risk of another business accidentally or intentionally snooping on data that is not their own.

Often this security risk is what keeps people away from the cloud altogether, but there is a way around it…

Dedicated Instances - private Cloud:
Pro: Security
Cons: Price

If security is a concern in your organization,
if you have to meet compliance standards,
hardware, data storage, and network is designed to provide high levels of security that cannot be accessed by other clients in the same data-center.
Customization for a dedicated framework - One size does not fit all
Yes, the price tag is larger.

References:
http://blog.fpweb.net/moving-to-the-cloud-choosing-a-multi-tenant-or-dedicated-environment-for-your-business/#.VTVO3SFViko

What is a spot instance in AWS? (32 of 285 technotes for 2015)

Spot Instances allow you to name your own price for Amazon EC2 computing capacity. You simply bid on spare Amazon EC2instances and run them whenever your bid exceeds the current Spot Price, which varies in real-time based on supply and demand.

If using spot instances make sure applications in the server

start automatically, after the server restarts
saves data often so that there is no loss of data
handles termination gracefully.

Sunday, April 19, 2015

How to extract data from PDF files using Apache Tika? (31 of 285 technotes for 2015)

The Apache Tika™ toolkit detects and extracts metadata and text from over a thousand different file types (such as PPT, XLS, and PDF). All of these file types can be parsed through a single interface, making Tika useful for search engine indexing, content analysis, translation, and much more.

The code below shows a simple example as to how to use Apache Tika.

List of 25+ Natural Language Processing APIs - Mashape Blog

Opensource project to convert fixed length flat file or string to Java object? (30 of 285 technotes for 2015)

BeanIO is an open source Java framework for marshalling and unmarshalling Java beans from a flat file, stream, or simple String object.

http://www.beanio.org/

Tuesday, April 14, 2015

What is Forum Sentry? (29 of 285 technotes of 2015)

Forum Sentry is an XML Appliance

Industry’s only FIPS 140-2 and NIAP NDPP certified API Gateway product

The Federal Information Processing Standard (FIPS) Publication 140-2, (FIPS PUB 140-2), is a U.S. government computer security standard used to accredit cryptographic modules.
NAIP - National Information Assurance Partnership -

is a U.S. government initiative that looks at products in the information technology (IT) realm and ensures that they adhere to international standards
NDPP - Network Device Protection profile -

security requirements for a Network Device (defined to be an infrastructure device (as opposed to an end-user device) that can be connected to a network)

built-in support for data formats and communication protocols.
runs the FIPS certified ForumOS
user interface

CLI (Command Line Interface) - available via SSH
Serial Console -
Web Admin interface - via HTTPS

There is no monitor key board or mouse access.

Network Interfaces

MGMT - for Management traffic
WAN - for external traffic
LAN - for bridging to the internal network

What is XML Appliance? (28 of 285 technotes for 2015)

What is it?

XML Appliance is a special purpose network device - used to secure manage and mediate XML traffic.
Useed in SOA and COC (Cloud Oriented Computing)

Integrate on-premise application to off premise cloud hosted apps..

synonyms - SOA appliances, SOA gateways, XML gateways, cloud brokers.
Specialized - MOM (Message Oriented Middleware)
Originally hardware - but now mostly software and also virtualized - due to advances in computing speed.

Providers:

Datapower & Vordel - 1999 - Hardware
Layer 7 - 2002
Forum Systems - 2001
Managed Methods - 2005

Industry:

XML - Cross division and cross company data exchange protocol
XML appliance used in -

finance, telecommunications, government, energy, logistics and more
addresses SOA Governance -

how applications delivered as “services” can be shared with or called by other applications
addressed message security, availability and translation of data so that an application can call another application irrespective of the data format and security policies

Reference -

http://en.wikipedia.org/wiki/XML_appliance

Monday, April 13, 2015

Managed node fails to synchronize (27 of 285 technotes for 2015)

Access is denied for the getRepositoryEpoch operation on ConfigRepository MBean because of insufficient or empty credentials.

In Administrative console:

SSL certificate and key management -> Key set groups ->Select Key set group name -> uncheck the box for “Automatically generate keys”

Clear the Automatically generate keys option.
From the Key set groups -> check key set Group name and hit Generated Keys tab.
Click OK and Save to save the changes to the master configuration.
Stop the dmgr
On dmgr side delete the contents under wstemp, temp and config/temp folder from <profile_root>
Start the dmgr
Stop the Node/Server using stopNode/stopServer commands from the <profile_root>/bin of AppServer
Manually synchronize the node by running syncNode.sh from <profile_root>/bin, since security is enabled then please run following command

syncNode.sh <DMgr_hostName> <SOAP_PORT_of_DMGR> -username <username> -password <password>
Start the node and server.
Logon to Dmgr Administrative console and check the Node/Server availability.

Reference: http://www-01.ibm.com/support/docview.wss?uid=swg21405907

Sunday, April 12, 2015

What is Phone Gap? (26 of 285 technotes for 2015)

PhoneGap is a free and open source framework that allows you to create mobile apps using standardized web APIs for the platforms you care about.

http://phonegap.com/

What is Adobe LifeCycle? (25 of 285 technotes for 2015)

Adobe LiveCycle Enterprise Suite (ES4) is a

SOA Java EE server software product from Adobe Systems Incorporated used to build applications that automate a broad range of business processes for enterprises and government agencies.

LiveCycle ES4 is an enterprise document and form platform that allows capturing and processing information, delivering personalized communications, and protecting and tracking sensitive information.

It is used to deliver solutions such as

account opening,
services and benefits enrollment,
correspondence management,
request for proposal processes,
and other manual based workflows.

LiveCycle ES4 incorporates new features with a particular focus on mobile devices.

LiveCycle applications also function in both online or offline environments.

These capabilities are enabled through the use of Adobe Reader, HTML/PhoneGap and the Flash Player clients to reach desktop computers and mobile devices.

Saturday, April 11, 2015

How to extract data from PDF Files? (24 of 285 technotes for 2015)

Apache PDFBox is a good tool to extract data from PDF files.

A Very good discussion about PDF Box… its issues, concerns, advantages and disadvantages.

http://www.lawandsoftware.com/blog/filling-xfa-pdf-forms-using-pdfbox/

—

http://svn.apache.org/repos/asf/pdfbox/trunk/examples/src/main/java/org/apache/pdfbox/examples/fdf/PrintFields.java

Reference: .

Thursday, April 9, 2015

Creating Windows Service to start / stop Websphere Application Server(23 of 285 technotes for 2015)

To Remove Windows Service

Go to d:ibmwebsphereappserverbin
d:...>wasservice.exe -remove “name of service” (from services)

To Add new Windows Service

Go to d:ibmwebsphereappserverbin

d:.…>WASService.exe -add “<service_name>” -serverName “<server_name>” -profilePath “<profile_path>” -logRoot “<log_server_path>” -logFile “<log_file_path_name>” -encodeParams -stopArgs “-username <user_name> -password <password>” -startType automatic -restart false

eg:

WASService.exe -add “CTHIX_DMGR” -serverName “dmgr” -profilePath “D:IBMWebSphereAppServerprofilesCTHIX_DMGR” -logRoot “D:IBMWebSphereAppServerprofilesCTHIX_DMGRlogsdmgr” -logFile “SystemOut.log” -encodeParams -stopArgs “-username wasadmin -password Admin.1” -startType automatic -restart false

For more info - http://ibm.co/1HZv9b9

Configuring Active Directory in Websphere Application Server (22 of 285 technotes for 2015)

http://bit.ly/1FrSyza

Thursday, April 2, 2015

What are the short comings of SSL? (21 of 285 technotes for 2015)

SSL/TLS Shortcomings -

SSL is point to point - when the message reaches a peer server
SSL needs to be stripped off so that the message can be processed

SSL Limitations:

SSL is point to point - where as we need end 2 end security, as typical enterprise application has multiple layers.

SSL is transport layer security - not message level security - data in disk cannot be protected by ssl

HTTPS - does not support nonrepudiation well - critical for business

server can prove that a completed transaction was requested by a client (in case a dispute arises)

SSL does not support element wise signing - i.e. sign only the credit card details of the entire message.

What is SSL Accelerators? (20 of 285 technotes for 2015)

SSL acceleration is a method of offloading the processor-intensive public-key encryption algorithms involved in SSL transactions to a hardware accelerator.

Typically this means having a separate card that plugs into a PCI slot in a computer that contains one or more co-processors able to handle much of the SSL processing.

References:
http://en.wikipedia.org/wiki/SSL_acceleration

How to create a WSRR profile using a response file? (19 of 285 technotes for 2015)

Run:

D:IBMWebSphereWSRRbin>manageprofiles.bat–response WSRR_Node.txt

Sample response file - ‘WSRR_Node.txt’

create
profileName=test_WSRR
serverName=WSRRServer
cellName=WSRRCell
nodeName=WSRRNode
hostName=DEVAPPESB4
profilePath=D:/IBM/WebSphere/WSRR/profiles/test_WSRR
templatePath=D:/IBM/WebSphere/WSRR/profileTemplates/default.wsrr
enableAdminSecurity=false
wsrrDbAlreadyConfigured=false
wsrrDbCreateNew=false
wsrrDbDelayConfig=true
wsrrDbOutputScriptDir=dbscripts
wsrrDbType=MSSQLSERVER_MICROSOFT
wsrrDbJDBCClasspath=D:/IBM/WebSphere/WSRR/jdbcdrivers/SQLServer
wsrrDbSysUserId=testwsr
wsrrDbSysPassword=
wsrrDbHostName=10.20.30.40
wsrrDbServerPort=1533
wsrrDbName=DWSRRDB01
wsrrDbUserId=hixwsr
wsrrDbPassword=
wsrrDbSchemaName=WSRR
actDbSchemaName=WSRRACT
sibDbSchemaName=WSRRSIB
configureBSpace=true

How to create a Websphere Profile using a response file? (18 of 285 technotes for 2015)

Execute:

D:DataIBMWebSphereESBbin>manageprofiles.bat-response node.txt

Sample response file below - ‘node.txt’

create
templatePath=D:/IBM/WebSphere/ESB/profileTemplates/default.esbserver
profileName=TEST_ESB
serverName=ESBServer
cellName=ESBCell
nodeName=ESBNode
enableAdminSecurity=false
dbServerPort=51000
ceiDbName=event
ceiDbAlreadyConfigured=false
configureBPC=false
dbCreateNew=false
dbCommonForME=true
fileStoreForME=false
dbHostName=172.31.1.188
dbType=DB2_UNIVERSAL
dbName=CMNDB
dbDelayConfig=true
dbAlreadyConfigured=false
applyPerfTuningSetting=standard
dbUserId=testesb
dbPassword=
configureBSpace=true
configureBRM=false
dbJDBCClasspath=D:/Data/TESTESB/jdbc
omitAction=defaultAppDeployAndConfig,samplesInstallAndConfig

How to create a Websphere profile in command line and add as Windows Service? (17 of 285 technotes of 2015)

Step1: WAS – Application server Installation path

D: IBMWebsphereAppserver

Step 2: Go to WAS_HOMEbin location (D:IBMWebsphereAppserver) and run the below cmd

manageprofiles.bat -create -profileNameTEST_CP -hostName DEVAPPPORTAL1 -cellName cell01 -nodeName Node01 -serverName CPServer -enableAdminSecurity -false -winserviceCheck false

Step 3: Once the profile is created add as Windows Service.

From the WAS_HOMEbin location (D: IBMWebsphereAppserver) and run the below cmd

WASService.exe -add TEST_CP -servername CPServer -profilePath D:IBMWebSphereAppServerprofilesTEST_CP

What is XML Signature? (16 of 285 tech notes for 2015)

XML Signature or Digital Signature:

Provides end 2 end message integrity - gurantee
authentication information about msg originator

To handle SSL shortcomings the XML Signature is used.

XML Signature:

“hash” the message -

arbitrary stream of bytes and converts it to a single fixed-size value known as a digest.
A digest is a one-way process: it’s “computationally infeasible” to recreate a message from the hash
or to find two different messages which produce the same digest value.
common hash mechanism is SHA1, the Secure Hash Algorithm

(read more in the link below)

——————————–

To hash the message - we need a shared secret key which should be difficult to guess

use Kerberos
when 2 entities want to communicate
a central authority distributes “tickets” that contain a temporary session key

Or use public-key cryptography

private key, known only to the holder
public key, accessible to anyone who wants to communicate with the key holder
anything encrypted with the private key can only be decrypted with the public key, and vice versa
(read more about RSA in the link below)

——————————–

I want to send message M
generate a digest, H(M)
signature - encrypt the digest with my private key, {H(M)}private-key
you receive the message, M and the signature
you generate the digest, H’(M)
decrypt the signature using my public key getting the H(M)
If H(M)and H’(M) are the same, then we know that M is the same

References:

MSDN - http://bit.ly/1Hnqek1

Thursday, April 30, 2015

Wednesday, April 29, 2015

IAM

IAM Roles

*** This is a very good discussion found in StackOevrflow - with little modification for my notes - you can find the actual discussion here***

Tuesday, April 28, 2015

Monday, April 27, 2015

Saturday, April 25, 2015

Friday, April 24, 2015

Thursday, April 23, 2015

Wednesday, April 22, 2015

Tuesday, April 21, 2015

Monday, April 20, 2015

Sunday, April 19, 2015

Friday, April 17, 2015

Tuesday, April 14, 2015

Monday, April 13, 2015

Sunday, April 12, 2015

Saturday, April 11, 2015

Thursday, April 9, 2015

Thursday, April 2, 2015

* This is a very good discussion found in StackOevrflow - with little modification for my notes - you can find the actual discussion here*