LoadBalancing Windchill Cluster with Pen

This is my first technical topic. Through all my technical topics I am trying to concentrate on those technical issues related to Windchill for which you might not get a direct support. These topic are purely based on the experience I had. You, the reader may be surely having other observations and experience and I would welcome you to post a reply to the topic so that we share knowledge

Terms I am using LB – Load Balancer

In a Cluster most of the systems are configured to work with hardware LBs. But there are cases when we have to select a software LB. Most engineers turn towards Apache while doing it. When implementing Apache we had to do some extra work encountered some issues which shifted my mind from Apache. One of the things is when working on a Unix environment its not an easy job to share a partition over machines. This needs to be done since if Apache is the load balancer it needs the static files of the Application accessible locally. We need to spend extra space on pasting the files on LB. Secondly on Apache 2.2 setting the AJP worker to balance the load it does balance the load but it does it wrong. Every other page hit is sent to another slave. In short, it is unable to maintain the sticky sessions. This is a very critical issue.

So we went out to the open source market to check if there were any other options for load balancing through softwares. We discovered Pen. Pen is a load balancer which is pretty good at maintaining the sticky sessions. It also maintains a history of client for past 2048 hits to the server and redirects the client to same slave in case the client revisits. Hold on, don’t think about too much since can get rid of the history.

Pen does not take too much of space at all neither a lot memory. It load balances the HTTPS protocol unlike most of us when using Apache configure AJP workers to load balance that is in practice.

Where do you download Pen?

http://siag.nu/pen

Download the zip file , un-tar it and follow the below procedure to install

$export PATH=$PATH:/usr/sfw/bin:/usr/ccs/bin

In the above line I have added sfw/bin and ccs/bin to include gcc and make in the path, make sure you have the respective path on your machine

$cd /usr/ptc/Windchill_9.1/

$./configure

$make

Once done, you will have pen install and you will be able to see all the necessary files.

Ok now there is nothing much to configure after we install. Below is pretty much the command you need your LB to kick off.

$ sudo ./pen lbhost:80 slave1:80 slave2:80

Examples

$sudo ./pen 146.123.123.1:80 146.123.123.2:80 146.123.123.3:80

$ sudo ./pen 146.123.123.1:443 146.123.123.2:443 146.123.123.3:443

This will get started your load balancer working.

As we discussed earlier that Pen maintains a history of past 2048 clients and redirects them to the slave by looking up in the history. If that is acceptable to you then it is fine since the clients are as it is equally distributed. In some case it might happen that clients that are mapped to one slave are high in number and that slave might get overloaded.

Also Pen keeps a watch on the slave to see if they are available. If not it will redirect the client to the live slave.

The solution to that is pretty straight forward. You reduce the table size down to any number between 1 to 5. By doing so the server will redirect the upcoming clients alternately. It won’t mess up the load balancing activity here since it also simultaneously monitors the hit ratio to the slaves.

To achieve this configuration you need to add the –c option in the startup command

$ sudo ./pen –c 1 lbhost:80 slave1:80 slave2:80

Examples

$sudo ./pen –c 1 146.123.123.1:80 146.123.123.2:80 146.123.123.3:80

$ sudo ./pen –c 1 146.123.123.1:443 146.123.123.2:443 146.123.123.3:443

Now these are some basic configurations I have given that you will need. There are other options that you can check our on its website and use accordingly, which is best for your cluster.

Windchill and Weekends…..

‘Not’ talking about Windchill is one of the biggest challenges I face on weekends which I usually fail at. I write my blog articles usually on weekends….LOL.

After I posted my first article on my blog I posted its link on Facebook for my friends to read it. It was a Sunday when I posted it. One of friends commented on my post saying how I could get time to write this stuff. And the next thing he started to discuss an ongoing issue that we were working on user logins, on Facebook itself. I told him I got time because I was not thinking about work. He told me even if its Sunday Windchill kept running in his mind. Well I had to admit it did in my head too

I don’ know how but it’s the magic of Windchill that keeps your mind running through workflow of thoughts..throughout the weekend. Especially if it’s an unresolved issue. I have seen many engineers in PLM who go through the same situation. When it comes to Windchill I need not remind users about the ‘Logout Button’ discussion J which is still running in many minds.

Unfortunately my friends are also from PLM(My friends…I am referring to the situation not you J). That’s why when we meet the biggest thing we stress upon is not to discuss Windchill issues. But no matter what we start talking we land on Windchill. We were travelling for dinner in Pune and a TATA Nano was standing near us. This car is the cheapest car in the world which makes everybody so fond of it. Looking at it we peeped into the car to see how the dashboard looked like since this car is pretty compact. Within a fraction of moment we started discussing if TATA used Pro/Intralink 3.x or Windchill 9.1 to design this car and the discussion went further to discuss the features in Windchill 9.1(and some issues too ;)). We later realized ‘what’ we were discussing and stopped it immediately before we sounded like geeks to people standing around us. Believe me, after 30 minutes we started it again when we saw a Kirloskar Generator humming.

It’s not bad that you talk about work on weekends it’s just what aspect of work you discuss. Although we discuss lot of Windchill we never actually discuss any issues.

Initial Days...Struggling and getting to know Windchill

I began my career in PLM in 2008. I began with getting trained on Windchill 8.0 functionality. No doubt when I was being trained on Windchill I began to realize that this was a huge product and I knew it could take me years to develop myself as an expertise over this product.

When you are actually new to PLM the first thing you wonder ‘Does this thing really help?’…I was pretty confused when I began Windchill and with its vast range of features always made me think how is it actually time saving…which is PLM’s biggest advantage..reduce TIME…from conception to reality.

The best way to learn Windchill would be to start using it. You can’t realize its actual use without using it. When you work on it as an admin you need to be aware of its huge list of feature which is almost next to impossible. There are a few people who actually know all of its features. But that definitely is a long shot. Sometimes it actually confused me. What module serves what purpose?

That’s where I was so involved in a team activity since there are many engineers in your team who all together know different bits and pieces of the software. You can’t work if you cannot communicate. You can learn more by giving away your knowledge. Working with juniors was a always a treat since sometimes you used to find out something that would discover. That’s what I enjoyed. I also see a great deal of interaction on PTC users forum where each and every Windchill users gets together to share information.

Before joining PLM I had very less or experience on actual CAD softwares because I had a Computer engineering background. Initial times were funny for me when I used to hear things like wtparts, epmdocuments and it bounced off my head. I had my previous career developing applications in Java but this was an entirely new field for me.

When it comes to Windchill functionality it covers a very wide scope of any industry process which are into vivid business mostly manufacturing. Creating such a software which has many modules and binding them together so that they can work together is surely a tedious job. But once they get together , things move pretty smoothly. The challenge is to understand how these modules are coupled together. If you are a newbie in Windchill you need to know that it’s going to be at least 2 years down the line when you will get yourself acknowledged with this software completely. If you want to make it simpler you need to understand its actual business purpose. If you don’t have the actual process knowledge on which Windchill is based, even simple things can seem tough and time consuming.

There are hundreds may be thousands of engineers who actually use a one Windchill system. Maintaining the performance of the system and administering its background softwares mainly Tomcat and Oracle is one of main things to get acknowledged with. Very frankly it isn’t an easy job, since usage of the system varies hour to hour. Increasing the Memory and clearing the caches is not the answer to every problem. Sometimes it is actually the other way around. Performance tuning is many-a-times a time consuming activity. When we are into performance tuning we need to set our own expectations first. I had worked on an issue where in opening of folder took 2-3 minutes which was clearly unacceptable to anyone. Analysis was done on this issue for about 2 weeks and then the time was brought down to 5 seconds. The issue was escalated again saying ‘5 seconds’ is unacceptable which lead to many angry engineers who spent time on the issue. When performance tuning is done we need to take care if there is any other module that may get affected performance wise if changes are made of one module. We need to decide our priority and expectation.

One of the most important things I learned when working on Windchill was ‘Patience’. Windchill is such a huge product and there is only one log file which reports it. It just prints it out there and you are the one who needs to find the culprit. Mostly it happens that the same error occurs for different issues. I had encountered an error of ‘unknown webname‘ after installing an OOTB Windchill 9.1. I knew what had gone wrong and I instantly went ahead and wrote a technical document on it. Next day morning I had an email from one of the senior engineers stating that it was ‘wrong’. I immediately setup a meeting with him and showed him why I wrote this document. But after that he showed me 3 other documents which had the same issue reported. The thing to notice was that the root cause was different. So you even though the error is same the root cause was different.

I spent most of my time in PTC on server administration, working on Tuning performances and also Enterprise down situations. EDOWN is a worst thing that can happen to an engineer….especially if it comes on Friday J. When I used to get EDOWNs I used to be in tremendous amount from pressure from my managers and client and their managers. The challenge at this would be cope up with that pressure and get on with your job and fix the issue. I had several cases when client used to start a fight and get frustrated. I would assume this an obvious case that even they had a sword hanging on their neck. My worst EDOWN was when I had a French guy who was sitting in South Africa. He had no idea how to speak English and I had no French tongue and the EDOWN went on ‘chat’ me using Google Translate to figure out what that guy was saying. At a time I got frustrated with all that and asked him if he could wait till the EU guys came up. He said that he could lose his job if I did not help him fix it. I had no choice but to help him at that time.

After that I spent about 3 years in PTC working on various modules of Windchill and it was a wonderful journey.