Getting Technical - DataStage and SAP BW

Let's say you have an SAP Business Warehouse and you want to get data out of it on a regular basis without going through the extract and FTP route. That's where a tool like DataStage (from IBM) comes in.

DataStage is an ETL tool (formerly form Ascential) that has as an option a connector capability to SAP or to SAP's Business Warehouse. Where I work our dependent data mart (MS SQL 2005) receives about 90% of its data from the SAP Datawarehouse. We don't use DataStage to place dat a back into BW (yet) so this will only be about downoading.

What am I going to write about?
My job for almost a year now has been to migrate our existing SAP BW to DataStage to SQL implementation from an SAP BW vs. 3 to and SAP BI vs. 7 implementation. I am fortunate that those that did the original work (BW 3 to DataStage to SQL) had done their homework pretty well.

This multipart posting will be about the work I had to do during the migration (still ongoing) and the learning curve/pactches/ and discoveries along the way. My hope is that someone else facing this type of migration will benefit from my experience.

Technical issues:
Our existing BW is BW 3: The new platform is BI 7 running on an IBM Linux server (12 CPU) somewhere in Europe. Our DataStage is version 7.5.1.A running on a Windows 2003 server. We are using the SAP BW pack version 4.2.1.

My next post: Why you cannot run both BW 3 and BI 7 DataStage on the same server...

Working for a living

I've had a few jobs in my life, worked in a hospital, computer operator, various programming jobs, systems analysis, entrepreneur, CEO, CIO, business broker, various consulting gigs, systems architect to name a few. Now I am a DBA (officially a data warehouse lead) at a multinational.

You'd think after that stint as a CEO I would be onto other things but I did a reevaluation and came to the conclusion that I really liked getting my hands on systems, solving problems and learning new technology. Plus, I really don't travel that much on business anymore which really fits what I want to be doing really well.

So I am going to start interspersing a few musings of discovery of a more technical nature in with these blogs. Specifically I am going add some of those 'ah ha" moments that I have run across in my new job. There are a lot of people out there with a lot more specific knowledge on some of the tools I am using/learning but I often am left to my own devices to solve problems (even after extensive googling!).

So here's what I am working on right now and I'll start posting what my discoveries are as I can.

My current project involves migrating the ETL processes from a SAP BW 3 to DataStage to MS SQL 2005 data warehouse project from BW 3 all the way to BI 7. To give you an idea of scope of the data, we daily transfer about 4m rows in 20 - 40 DataStage extract jobs.

My current nightmare involves the pesky DataStage Server 7.5.1.A Open Hub Extractor interface so my next post will focus on a few insights. Stay tuned....