An ESB Must Support the following mentioned below - Application Awareness in ESB
Service Virtualization
Improved Manageability - all the things below can be managed centrally
style sheets
security
caching
routing
Monitoring and Managing messages as they flow on the bus
Routing messages
Converting Protocols
Transformation
Securing messages
Providing connectivity to application services via open and propriety interfaces
Monitoring & Managing - loginformation about arrival path, contents of the msg based on filters
ESB general monitoring
track messages as they flow through the Enterprise - determine where time and resources are being spent
remote monitoring of Datapower appliances - using SNMP, WSDM, WS-Management and proprietary SOAP API.
for end to end monitoring for the ESB as well as the services is to - integrate Enterprise Monitoring software through std protocols like SNMP and ARM - like ITCAM for SOA
ESB service level management (SLA) via monitoring- this is one of the most important
throttle (reject) and shape(delay) traffic based on patterns below
Prioritization
Count Monitors
Duration Monitors
protects the backend application resources when.
application latency reaches a threshold
processing latency reaches a threshold
Routing - is required the 2 reasons mentioned below
Quality of Service - able to prioritize some message based on the the SLA
Support of specific functionality or affinity -
some functionality provided in a particular version
session may exist on a particular destination - requiring affinity based routing
Different types of Routing
Content based routing
Context based routing
Aggregation and Disaggregation
Protocol Conversion -
need to move messages from one protocol to another
web based scenario's are asynchronous, but back end systems are transaction oriented and want it to be synchronous. So state management responsibility is also there for the mediator.
if Websphere ESB is used - to convert protocol and transform and route message - it in general degrades performance
Message Transformation -
XML-to-XML Transformations
HTML/SOAP to XML Transformations
Binary Transformations
Content Enrichment and Filtering
Securing Messages -
Though it has a large set of Security services - but a fairly small set of Services should be available for an ESB
Support for standard WS-Security protocols as defined in the WS-I Basic Security Profile
Support for credential mapping across multiple protocols and transports
AAA
Connectivity -
References:
Datapower Architectural Design Patterns - Integrating and Securing Services Across Domains - http://www.redbooks.ibm.com/redbooks/pdfs/sg247620.pdf
Datapower typical security - solves the issues below
Cross Enterprise Inter-operation -
Federated Interoperability -
Human and automated service invocations -
Dynamic service binding -
Global architecture layers impact -
Datapower as a XML firewall
While developing internet facing applications, the concept of DMZ is very important - this is the place where a hardened bastion host is placed between 2 firewalls.
Below is the standard topology - without Datapower
The WebServer - does almost nothing
Most of the work is done by Webservices Gateway
Using It is the most hardened device available in the market.
the role of Datapower within the DMZ is that it needs to be able to stop any incoming request and provide authentication and authorization - depending on Business Requirements.
Eliminates the Web Server and the Web Services Gateway
More secure - easy to manage
Below are a few of the protection which Datapower provides -
XDOS
well formedness
verify digital signature
signing messages
implementing service utilization to mask internal resources via XML transformation and routing
encrypting data at the field level
Three basic types of Firewall
Static backend
Dynamic backend
Loopback
Recommendation: All internet-facing systems that provide for inbound Web services, requests should use Datapower as their XML firewall, even when performance or security is considered unimportant.
Recommendation: DataPower should be used as the policy enforcement point for Web services authorization. It should interact with a central policy decision point, such as Tivoli Access Manager.
Auditing and compliance for SOA: Compliance Automation)
User provisioning: WS-Provisioning/SPML 2.0.
XML-level protection enhances SOA security management.
This simply maps the different steps of the DataPower AAA framework to specific IBM products and standards that may be used in each of these steps.
TAM - provides Authentication & Authorization
IBM Tivoli Access Manager ( now IBM Security Access Manager) handles the authentication and authorization part of your IAM infastructure.
TFIM - provides federated identity between organizations
IBM Tivoli Federated Identity Manager allows for federated and web Single Sign On. It can be used with ISAM, for example in a scenario that ISAM delegates the authentication part to TFIM for certain resources/cases.
ISAM does not speak SAML by itself, but it can leverage TFIM that does.
TAM and Datapower -
In order to connect to TAM from DP, a TAM client must be configured in the DP SOA appliance.
SAML, a user can login to one system in an environment, and then will be able access to other systems in that environment without needing to login again (until the web browser session is ended).
XACML PEP/PDP -
eXtensible Access Control Markup Language
Policy Enforcement Point/ Policy Decision Point
The standard defines a declarative access control policy language implemented in XML and a processing model describing how to evaluate access requests according to the rules defined in policies.
Kerberos & SPNEGO
a protocol for authentication
uses tickets to authenticate
avoids storing passwords locally or sending them over the internet
involves a trusted 3rd-party
built on symmetric-key cryptography
Kerberos is normally deployed in a client/server environment. It is rarely used in web-applications and thin client environments.
Because of this, SPNEGO comes to the rescue. It stands for Simple and Protected GSS-API Negotiation Mechanism, which provides a mechanism for extending a Kerberos based single sign-on environment to web-applications.
To view the contents of any phase, enable the multi-step probe for the firewall or Web Services proxy - employing a document processing policy containing a AAA action
AAA Action :
AAA Action is a Datapower object - that references a specific AAA policy. This is the way to add (bridge) a AAA policy to any of the services mentioned below.
Since WebService requests have to travel multiple security domains, the credentials used in the inbound to a boundary server, often require transformation, before reaching the recipient.
Types of Transformation:
Technology -
changing a credential from one type to another
eg: sender might use Digital Signatures, while the receiver might use Username token.
Naming -
the name that represents a user might change, similar to Relationship in Process Server.
eg: your identity to IBM can be your serial number, but to bank it can be bank account number.
To Resolve this -
Custom developed ad-hoc code -
Datapower can perform complex transformations - using built in functions as well as custom transformations.
Leverage a product like TFIM (Tivoli Federated Identity Manager)
Datapower can also connect to products like TFIM.
Rule of THUMB:
Use Datapower - for simple credential transformation
User TFIM -
for complex credential transformation
when transformation occurs in multiple places like Datapower and WAS.
Datapower consumes messages in the network speed, and can remove the WS-Security overhead and pass the clear message to back-end server and vice versa.
Felxibility -
Datapower supports more of WS-Security than of any Application Server - maybe Websphere will catchup the near future, but as of now they have a complete coverage.
Datapower fundamentally includes XSL processing engine - which helps in easy Transformation of messages.
By offloading the WS-Security processing to Datapower, we eliminate the need to configure and re-configure the back end server.
Example of Datapower usecase:
Client who want to use WS-Security but find that their Application Server, does not fully support this, should evaluate Datapower.
Clients expecting to frequently change WS-Security settings,
Why WS Trust and Secure Conversation? WS-Security adds enterprise-level security features to SOAP message exchanges, but with a substantial performance cost. WS-Trust builds on WS-Security to provide a way of exchanging security tokens, WS-SecureConversation builds on WS-Security and WS-Trust to improve performance for ongoing message exchanges
What is the performance issue with WS Security? WS Security uses asymmetric keys - that is public and private key pair, which require larger keys and complex processing while decrypting messages vs single secret keys known to the both parties. WS Conversation uses WS Trust (+ WS Security) to use only a single key pair to communicate between client and server. What is WS-Trust? WS Trust is a WS based standard for Requesting / Receiving the tokens
Function 1 - Implement the STS to
issuing
renew
cancel
validate Security Tokens
Function 2 - Support brokering Trust relationship
What is STS (Security Token Service)?
An STS is a web service that implements a simple interface defined by the WS-Trust specification. The operations supported are issue, renew, cancel and validation of Tokens.
WS Conversation - is a standard which allows symmetric encryption to be used ongoing exchange of messages between client and the server. -----------------------
How WS Trust is established?
Consumer Requests a Security Token (RST) to the STS
STS returns a signed token to the Consumer - Request Security Token Response (RSTR)
Once the token is received - the requester can present it to multiple services.
DB is good solution for a low traffic application:
If you are building a simple application with low traffic, there is
something to be said about keeping another component out of your system. It
is very likely not using a message bus is the right answer for you.
However, I would suggest building your system in a way you could swap out
the database based queue system for middleware solution. I agree with the
article. A database is not the right tool for queue based system, but it
may be good enough for you.
Advantages of Queue based systems:
Queue based system like RabbitMq are built around massive scale on moderate
hardware. Their architecture is able to achieve this by avoiding processes
which make ACID compliant database system slow by their nature. Since a
Message bus -
only needs to ensure a message is stored and successfully processed,
it doesn’t need to bother with locking and writing transaction logs.
Both of these concepts are absolutely required for an ACID system but are
often a cause of contention.
Issue with DB:
Performance wise it comes down to, you have a SQL table. Lots of reads and
lots of writes. Both require some sort of locking to update rows, pages and
indexes. Your polling mechanisms is constantly locking an index to do look
ups on it. This prevents writes from happening, at best they are queued.
The code doing the processing is also locking to update the status on the
queue as they complete or fail. Yes, you can do query optimization after
optimization to get this to work, or you can use a system specifically
designed for the work load you are asking.
A RabbitMq eats up this type of workload without even breaking a sweat.
On top of that you get to save your database from the workload giving it
more room to scale doing other things.
Polling:
Another thing to consider is most queue system typically do not use a
polling technique (some allow for HTTP, but recommend to avoid using for
the receive side). RabbitMq uses network protocols specifically designed
for message buses such as AMPQ.
DB is good solution for a low traffic application: If you are building a simple application with low traffic, there is something to be said about keeping another component out of your system. It is very likely not using a message bus is the right answer for you.
* When you’re done, hit Ctrl + C on the keyboard, type :wq and hit Enter.
* Restart the server
* Then start vnc server again.
vncserver
In the vnc client, give public DNS plus “:1” (e.g. www.example.com:1) or private ip address. Enter the vnc login password. Make sure to use a normal connection. Don’t use the key files.
The declared package “org.jboss.as.quickstarts.helloworld” does not match the expected package “main.java.org.jboss.as.quickstarts.helloworld” HelloService.java /jboss-helloworld/src/main/java/org/jboss/as/quickstarts/helloworld line 17 Java Problem
Resolution:
To resolve this issue.
Right click on the the project → Properties → Java Build Path → Source
JAXP (Java API for XML Processing) is a rather outdated umbrella term covering the various low-level XML APIs in JavaSE, such as DOM, SAX and StAX.
Create a SAX Parser or DOM Parser and then PArse the data, if we use DOM, it may be memory intensive if the document is too big. Suppose if we use SAX parser, we need to identify the beginning of the document. When it encounters something significant (in SAX terms, an “event”) such as the start of an XML tag, or the text inside of a tag, it makes that data available to the calling application.
Then Create a content handler that defines the methods to be notified by the parser when it encounters an event. These methods, known as callback methods, take the appropriate action on the data they receive.
JAXB (Java Architecture for XML Binding) is a specific API (the stuff under javax.xml.bind) that uses annotations to bind XML documents to a java object model.
Bind the schema for the XML document.
Unmarshal the document into Java content objects. The Java content objects represent the content and organization of the XML document, and are directly available to your program. After unmarshalling, your program can access and display the data in the XML document simply by accessing the data in the Java content objects and then displaying it. There is no need to create and use a parser and no need to write a content handler with callback methods. What this means is that developers can access and process XML data without having to know XML or XML processing
Select the server.xml specific to your server and edit it to add/modify/delete the attributes @initialHeapSize and @maximumHeapSize of the element <jvmEntries> The value of these attributes is the heap size in MB.
Adjust paths in the commands below to match your environment
Fix (manual)
Run this if you just want to test if this helps
** I placed the sqllite3.exe in the same folder where we have wc.db
D:svn_pmp.svn>sqlite3.exe wc.db SQLite version 3.8.10.2 2015-05-20 18:17:19 Enter “.help” for usage hints. sqlite> select * from WORK_QUEUE; 812|(file-commit pmp-services/src/main/java/com/hixapi/pmp/service/plan/impl/mapping/PlanRateMapping.java) sqlite> delete from WORK_QUEUE; sqlite> .quit
Fix (automated)
If previous step worked for you, consider automating the process with these steps
Go to your .svn folder, e.g.
D:svn_pmp.svn
Copy sqlite3 shell tool there
Create a fix-svn.bat file in that folder
Next time you need to fix it, just run the shortcut on your desktop
Insert scripting code, and adjust paths
"d:svn_pmp.svnsqlite3.exe" wc.db "delete from WORK_QUEUE" "C:Program FilesTortoiseSVNbinsvn" cleanup "D:src"
IBM products has a license cost and a support cost.
JBoss does not have a License cost - but has a subscription cost. Red Hat JBoss Fuse subscription is over $823K less than IBM WebSphere Enterprise Service bus at list price—or just 7.3% of the IBM cost.
Interest Over Time:
Click on the link to see how Google Trends shows the interest over the time between the two products and the latest Integration technology - Apache Camel (which is used in JBoss Fuse) Websphere ESB vs (Jboss Fuse & Apache Camel) (live link)
Sample below - the blue line shows decrease in interest IBM WebSphere technologies
Support:
RedHat JBoss Fuse can be used in any customer locations.
The same is applicable for IBM ESB (except IBM WESB - Registry Edition or Retail Edition)
Developer Tooling:
RedHat JBoss Developer Studio is used for all Middleware development.
Similarly IBM Websphere Integration Developer used for all development purpose. IBM tooling is more matured - but Red Hat is catching up.
Development:
JBoss Fuse development is extremely flexible - developers can swtich between graphical and xml views. Developers can can write a Apache Camel entirely in Java.
IBM does not encourage switching between gui mode and xml mode.
Other Tools:
With Red Hat Subscription, developers can choose to explore all the RedHat JBoss Middleware platforms like RedHat JBoss Data Grid and BRMS etc.
This is not possible with IBM.
Building Integration using patterns:
RedHat JBoss uses Apache Camel - open source, out of the box, standards based integration patterns based out of the book - Enterprise Integration Patterns by Gregor Hohpe and Bobby Woolf.
IBM WESB does not allow pattern based development
Server deployment footprint:
RedHat JBoss Fuse requires 100mb free disk space and 2GB RAM. This can be even tuned and optimize.
IBM in geneal required 6GB RAM to run IBM WESB (including IBM Application Server).
Embedded Java Development:
Use the Camel core component of Red Hat JBoss Fuse to embed routes in Java applications. This kind of feature is not available in IBM ESB.
Reliable Messaging:
RedHat JBoss Fuse uses ActiveMQ - an open source software with a very small footprint. ActiveMQ.
It has several other benefits over IBM Websphere JMS. Please see link below for more info.
Transport Bindings
RedHat JBoss Fuse supports all the different types of transport binding including HTTP, JMS, HTTPS, FTP.
On top of that Camel provides 125 connectors to varied systems.
Fuse Fabric is a runtime environment that leverages capabilities of Apache Karaf, the container underlying JBoss Fuse, to provide centralized configuration and provisioning capabilities. In brief, Fuse Fabric provides the ability to create container instances running locally or remotely, including on cloud environments like Amazon EC2 and Rackspace. You can then create and apply Profiles that define what OSGi bundles and associated configurations (name / value pairs) to apply to each managed container.
If required adjust the number of processors and memory Datapower uses.
Static IP:
Do you want to install Wizard? y
Step1: Do you want to Configure network interfaces? y Do you have this information? y Do you want to configure the eth0 interface? y Do you want to enable DHCP? n Enter the IPv4 address for the interface in CIDR notation: 10.0.0.124/24 Enter the IPv4 address for the default gateway: 10.0.0.1 Do you want to configure the eth1 interface? n Do you want to configure the eth2 interface? n Do you want to configure the eth3 interface? n
Step2: Do you want to configure network interfaces? y Do you want to configure DNS? y Enter the IP Address of the DNS Server: 10.0.0.1 (this is the ip address of my default gateway)
Step3: Do you want to define a unique system identifier? y Enter a unique system identifier: data123
Step 4 - Do you want to configure remote management access? y Do you want to enable SSH? y Enter the local IP address : 10.0.0.124 (this is the static ip for the DP) Enter the port number: 22
Do you want to enable WebGUI access? y Enter the local IP address : 10.0.0.124 (this is the static ip for the DP) Enter the port number : 9090
Step5-7 - no
Saved the configuration
Post Installation: Now goto internet browser and type the url
Step1: Do you want to Configure network interfaces? y Do you have this information? y Do you want to configure the eth0 interface? y Do you want to enable DHCP? y Do you want to configure the eth1 interface? n Do you want to configure the eth2 interface? n Do you want to configure the eth3 interface? n
Step2: Do you want to configure network interfaces? y Do you want to configure DNS? n
Step3: Do you want to define a unique system identifier? y Enter a unique system identifier: data767
Step 4 - Do you want to configure remote management access? y Do you want to enable SSH? y Enter the local IP address : 0.0.0.0 (this is the default ip) Enter the port number: 22
Do you want to enable WebGUI access? y Enter the local IP address : 0.0.0.0 (this is the default ip) Enter the port number : 9090
Step5-7 - no
Saved the configuration
Post Installation:
cmd -> show int this will give the ip address
Now goto internet browser and type the url
https://10.0.0.16:9090 If there is any issue with the DP Portal not working, login to CLI and type the command below
→ show int
If you dont see the ip address for eth0, type the commands below
→ co → ethe eth0 → show
Check if the above value of ip-config-mode is static / dhcp. If static, set it to dhcp
→ ip-config-mode dhcp → exit → write mem → exit → show int
Navigate to the VMware Workstation folder in Program Files.
Double-click vm-support.vbs support script.
When the process completes, you are presented with a message informing you that a file was created in a folder calledvmware-support in your host’s %TEMP% folder. The compressed support bundle name begins with vmsupport and includes a date stamp for month, day, and year.
Virtual WiFi is a technology that virtualizes your network adapter much in the same way VMWare virtualizes your entire operating system. Once virtualized, you can basically convert one physical wireless adapter into two virtual ones
Default Gateway - is a the device that passes information from the local subnet to some other subnet (local network to internet)
In home, the home router is assigned the default gateway ip address - 192.168.1.1
What is a CIDR Notation?
First address of a network, followed by a slash character (/), and ending with the bit-length of the prefix. For example, 192.168.1.0/24 is the prefix of the Internet Protocol Version 4 network starting at the given address, having 24 bits allocated for the network prefix, and the remaining 8 bits reserved for host addressing
What is a Subnet?
The practice of dividing a network into two or more networks is called subnetting.
Subnets may be arranged logically in a hierarchical architecture, partitioning an organization’s network address space into a tree-like routing structure.
What is a Subnet Mask?
A subnet mask is a screen of numbers used for routing traffic within a subnet. Once a packet has arrived at an organization’s gateway or connection point with its unique network number, it can be routed to its destination within the organization’s internal gateways using the subnet number.
What is a DHCP Server?
Where is a DHCP Server located?
DHCP server is located in the Router. If you do ipconfig /all you will see any entry mentioning the DHCP entry.
Goto your browser and type the ip address of the DHCP Server -> it will take you to the router admin console
Most likely it will be the same as the default gateway.
What is a DNS Server?
Domain Name Servers (DNS) are the Internet’s equivalent of a phone book. They maintain a directory of domain names and translate them to Internet Protocol (IP) addresses. This is necessary because, although domain names are easy for people to remember, computers or machines, access websites based on IP addresses
Once your import task is completed, you can use the ec2 run-instances command to create an Amazon EC2 instance from the AMI generated during the import process.
Changing Market - iBPMs include social, analytics and cloud in their offering
For different factors for iBPMs are
Composition of intelligent, process-centric applications
Continuous process improvement (CPI)
Business transformation
Digitalized process
Appian
what is happening right now within their organization, markets and customers, and accelerate the translation of that awareness into action and business outcomes
social centricity
Short Comings
does not have full CEP(Complex Event Processing) and analytics capabilities
less support for top down transformation.
Pega System
Largest pure play vendor
can be leveraged for large complex business scenarios
advanced operational decision management capabilities, predictive analytics and IoT integration
Short Comings
did not focus on low end mid market
finding resource is difficult
IBM BPM
Steadily moving to cloud
enabling mobile, social, cloud and big data while driving efficiency and optimization into end-to-end processes.
starting to handle - more-agile, lightweight projects.
