SFTP vs FTPS: tough choices

Las week, I had to design in a hurry a secure file transfer mechanism between two DMZs on a zero budget which, in a nutshell, meant reusing the Windows servers that are already there, and not purchase any third party software.

I had to choose between using SFTP, a nice protocol, and FTPS, which I've been comparing to a bastard child for years.

I don't like FTPS mostly because it's a patch on FTP. For one, FTPS is harder to firewall than SFTP; it behaves exactly like the standard FTP with a control and data connection, the difference being that TLS is used to encrypt them. Like with standard FTP servers, the server must be configured with a fixed range of passive ports, and the firewall must let these ports through. Why? Because the firewall has no way of knowing what dynamic port has been assigned to a passive data connection... it can't sniff it out the control connection either, as it's encrypted!

Even though it's not exactly what I would call an elegant protocol, is FTPS actually easy to work with? The answer is yes: I was able to install IIS 7.5's FTP publishing service in 2008 R2 and have an FTPS server working within minutes. That is good enough. And in IT, good enough is, well, Good Enough.

So, here are my thoughts:

If your server will be hosted on any kind of Unix, choose SFTP. It has been built-in with OpenSSH for years. The drawback of OpenSSH is that it doesn't support virtual users, and this can make high availability tricky; you'll need to synchronize /etc/passwd entries, even if using AD authentication.

On the other hand, if you will host the service on Windows, you might be better off going with FTPS as it is included with IIS 7.5 and there is even high availability that is possible. To support SFTP on Windows, you either need to install unsupported open-source software (unacceptable in many secure, enterprise environments) or purchase a third-party product such as WS_FTP Server (which carries a premium if you need SFTP functionality).

As for CLI clients that support automation, no matter the platform you use, there are plenty to choose from. For SFTP, on Unix just use the sftp command and on Windows, try Putty's excellent psftp.exe. For FTPS, I suggest you try cURL which is multi-platform on Unix and Windows.

So, to conclude: SFTP if using a Unix server, FTPS if using a Windows server.

In my case, I'm going with FTPS.

O.

Gabriel Consulting Group survey on Oracle and HP-UX

As many HP-UX admins still read this blog, I thought I'd post this. GCG is running a survey to have some insights on what you're thinking about Oracle's decision to stop developing products on Itanium, and Oracle in general:

http://survey.gabrielconsultinggroup.com/limesurvey/index.php?sid=73634&lang=en

I got this link from an article that Dan Olds posted on The Register.

I takes maybe 10-15 minutes to answer the survey and I think it is worth it, as the results will no doubt end up being published by HP somewhere down the road. Even though I'm sure they're independent, the questions and tone of the survey are not, er, I'd say, totally objective. I answered it not as an HP-UX admin (which I'm no longer), but as a systems architect for an enterprise that runs a mixture of HP-UX, AIX, Solaris, and Windows. So I tried to stay unbiased. You should do the same.

O.

PI DataLink Server and Excel Web App: A wedding cake dilemma

photo by Michael Prudhomme

The project I'm attached to has in its list of technical requirements the installation of Excel Web App (EWA) along with PI DataLink Server (DLS). It is not clear what the customer intends on doing with it, but my guess is that it will be used to show PI data to end users using a web-based interface.

The DLS manual describes four user roles, two of which are directly related to Excel: a publisher and a reader. This pretty much resumes what it is designed for: some people, who are PI experts, develop and publish workbooks using a real Excel with PI DataLink, while common end users read them, using a browser. This apparently read-only nature of DataLink Server (which I need to confirm) is an important one, as from my understanding, it is positioned to be a simple web reporting platform.

I've recently had some time to experiment with these web features to try to predict what the developers will end up doing in the long run. I also had the hope of leaving the marketing pitch to marketers and finding what were the real advantages of going in that direction instead of sticking with a deployment based on the standalone Excel application.

I'm not an Excel whiz kid, and I'm even less a SharePoint expert. That being said, after a few mishaps, I've managed to make a proof of concept with DLS and EWA using the most dummy report I could build with my limited knowledge of PI:


The wedding cake dilemma

I'm glad to announce PI Datalink Server works as designed within the Excel Web App. However, when playing with it, I couldn't stop thinking about a three-layer wedding cake. Why? Because you see, pitting EWA against standalone Excel is like comparing that wedding cake to a slab of brownies. Both will easily feed dozens of people, but the wedding cake will take longer to assemble, be more expensive, and each layer will need to be supported by the one underneath (I also think the brownies will be tastier, but that is beyond the scope of this article).

I had no doubt that the combination of the three layers consisting of DataLink Server, Office Web Apps and SharePoint involved lots of other subsystems too. This presentation done by Microsoft last year confirmed my suspicions. IT Operations would have a hard time supporting all that if the dependency hell between all those subsystems ever hit the fan. Understandably, as a systems architect, I wasn't very comfortable in greenlighting the use of DataLink Server at first glance. Is it safe to assume that if an architecture is made like a wedding cake, it better offer something big in return or else it's not worth it?

I think that in that particular case, it will be worth it if your experts use PI DataLink a lot and they need to deploy ad hoc reports quickly to a controlled (i.e. not massive), read-only audience.


Using EWA and DLS for ad hoc reporting

The ugly sample report pictured above is what I would call an ad hoc report: It's a quickie, made in a hurry to fulfill an unexpected business need. These can be done in a matter of minutes and published as a web spreadsheet to be consumed by users who have no technical knowledge of PI. There is no need for these users to have Excel on their client, as everything runs in a stripped-down version of Excel straight in the browser. This could prove extremely useful when dealing with mobile devices in the future as I don't expect Excel and DataLink to be running on the iPad anytime soon.

Furthermore, since you don't have a bunch of standalone Excels running around in the wild, you don't have to:

1. Ensure all users have the correct Excel version;
2. Install PI Datalink on each of these Excels and maintain this installed base which can be substantial;
3. Deal with the security hassles of opening up network access to the PI infrastructure to every laptop in your WAN (you only need to open it to the server running DLS).

Interesting. One might expect a lot of reports to be created that way.


Preventing ad hoc report sprawling

Now comes a question: what do we do to prevent "ad hoc report sprawling"?

I think that ad hoc reports should be deployed to VIP users as prototypes, until the time comes to move to something better if they ever need to reach a wider audience. By "something better", I'm talking about a dedicated reporting system such as Crystal Reports for the kind of reports that pull data not only from PI, but also from AF and other sources. The kind of reports that are read daily by people who make business decisions based on their contents. The kind that end up on a printer, to be read to/by upper management.

These official reports should still be designed, deployed and stored on a dedicated platform. Why? Because:

1. EWA and and DLS have their limits; my understanding is that they can pull out data only from PI points, not AF (on the other hand, there are ways to combine web parts with DataLink Server, but I'm not good enough to try that out);
2. I also have a feeling that using EWA as a reporting solution might cause a performance impact both on your SharePoint and PI infrastructure as nothing will prevent John Doe from pressing CTRL-ALT-SHIFT-F9 (in caps, of course) all the time to be updated on the second. It's much slower on DLS than within the real Excel, so I think there is a performance hit. This impact needs to be evaluated, and thus why I talked about a controlled audience above.
   

Conclusion

The possibility of deploying ad hoc reports to read-only users who don't need to have Excel at all is the main advantage I've seen up to now to deploying an architecture based on PI DataLink Server, Excel Web App and SharePoint. However, as this might be a complex solution that your IT Operations will need to take care of in the long run, you need to be sure you really need it.

O.

Am I off the track on this? Have any comments? Please post below and I'll be glad to write an update to this article.

Oracle dumps Itanium. -1 for HP-UX, +1 for Integrity

I'm no longer involved with HP-UX and Itanium but I've been an HP-UX geek for 10 years, and some of the readers of my previous blog (aptly named Technocrat-UX) were following me specifically for my comments on that market. I'm no analyst, but anyway my advice is free so here are my thoughts on this story which was unraveled today.

I'm not surprised about Larry's decision to stop development of Oracle on Itanium. It was just a matter of time before Oracle would try different stunts and measures - any measure - to try to save their SPARC platform and lock in customers. This one is a desperate measure indeed. I know a lot of SAP system administrators who won't be delighted to learn this. Some have been claiming for a while that Oracle is the new CA, and this couldn't be more true. If I was still an HP-UX admin, I'd be directly targeted by this decision. But I wouldn't say "Fuck HP". I'd say "Fuck Oracle", big time.

How does that look for HP-UX? Without an enterprise RDBMS, not good. Not good at all. But all is not over for Integrity. Rob Enderle picked up the story today and revealed some interesting information that was shown to him under an NDA:

Unfortunately for Oracle, HP just had a massive analyst event and in the server break-out had showcased under NDA the future for Itanium in new products. While I can’t share that future, it is NDA, and for those of us in the session there was no doubt that Itanium is going to continue. More importantly, the changes being made should make it vastly more cost effective than anything Oracle can announce on SPARC. You’ll understand what I mean in a few months, or if you have an HP relationship, ask HP what I’m talking about and you’ll have a big “ah hah” moment. But you won’t be able to share it any more than I can.

I've been thinking about his statement, trying to read between the lines. Here is my own speculation of what may be ahead. Note here that he's talking about the Itanium platform, there is no mention of HP-UX anywhere in his post. What can I make of this?

Here is what I know:

1. Enderle says that the new platform will be "vastly more cost effective than anything Oracle can announce on SPARC".

2. I bet that Microsoft are probably annoyed by Oracle (sorry, no time this evening to find an article to back this up).

3. I've learned from a trusted source (without signing a CDA) that the DL785 will retire and only the DL580 and DL980 series will be left. Which anyone can deduct from HP's web site, there is currently no G7 offering of the DL785.

...and here is what I predict:

1. Microsoft will be looking for an enterprise-level platform to harness MS-SQL, which has become over the years the "other" enterprise RDBMS.

2. All my current architecture projects are based on MS-SQL and I've learned a little about its licensing recently. This is software priced per CAL or per processor/socket (your choice), and each processor costs - well, costs a lot of money. Customers with thousands of users will want to get the most bang out of every processor they use. Does that sound like a return of Windows and MS-SQL to the Integrity platform? Hard to say if MS keeps a similar pricing with Tukwila's four cores, but it is possible. Microsoft could offer this as a vertically integrated solution pitched to customers who are currently relying on Oracle (or IBM's) solutions.

3. To cater to these mid-size Microsoft customers who aren't interested in blades (let alone Superdome 2s) for a reason or another, HP will release something like the rx5800 and rx9800 which will be based on the industry standard components of the big Proliants - namely, the 580 and 980. These are reliable and huge workhorses. By swapping only a few components, they'll save plenty and be able to offer these servers at a small premium over the x86 versions.

4. As for HP-UX? While the outcome isn't clear, I frankly don't expect Oracle to really stop releasing their RDBMS on HP-UX; I'm sure there are plenty of customers left and some bean counter at Oracle will realize the high risk of loosing them forever if a migration to SPARC is shoved down their throats. If they keep their stance, their loss. I haven't cared about Oracle for a while now.

These are my thoughts.

O.

Building a PI Lab for SharePoint 2010 and Excel Web App (Part 2)

Didn't have as much time today, but here are my findings:

1. A standalone ProcessBook installation requires a huge dependency package. So if you intend on deploying this on hundreds of PCs, better think of it. It should be installed on a TS, or deported using Citrix. Of course, for large scale deployments, it is better to plan using Web Parts over ProcessBook...

2. PI Datalink 2010 is supported on Excel 2010 32-bit only. It's documented in the Release Notes but as usual I didn't RTFM, and had a 64-bit installation. The PI-DL installer doesn't tell you anything about this, and it results in the add-in not being installed in Excel as it should be.

3. If SharePoint was a blind date, I think I would stay polite, possibly pay the whole bill, then say goodbye with a kiss on the cheek... i.e. leaving all options open while sending a clear message at once. Web page editing is sluggish, which is unacceptable in 2011 for a web application and I don't care how slow the back-end is. And it is hungry as hell. Even a vanilla installation revealed a lot of clunkiness with some "oops" error messages and dead links. Bottom line is I don't like SharePoint as of now, but I'll have to get used to it.

4. The drawing on my first post isn't right. I'll need to fix it in an update.

O.

Building a PI Lab for SharePoint 2010 and Excel Web App

I finally had some spare time today. No meetings, first time in a while. I had the chance to continue building my PI Lab and try to see how I can use SharePoint 2010 and Excel Web Services. My intentions are to harness the Web App version of Excel as much as I can, so that users can get PI data without needing Excel or PI DataLink at all. Developers will also be able to publish some pre-formatted reports on SharePoint.

I reinstalled almost everything from scratch to start afresh. It's not completed yet but here's what it should look like. Maybe it could inspire some of you. As a bonus, it shows the interactions between various OSISoft layers, which is not always clear to a neophyte such as I.


This is by no means what we'll have in production, everything has been installed with standard and typical (i.e. non-secure) settings and it is used to test and evaluate the interaction between Microsoft and OSISoft components only.

At the bottom, you have your honest-to-goodness vCampus PI System. It doesn't have any interfaces, but can provide some mock points such as good'ole CDT158 and BA:TEMP.1.

In the middle, a default dumb SharePoint 2010 server has PI-SDK feeding PI Data Services, and PI DataLink Server 2010. They, in turn, feed Web Parts and Office Web Apps (which, in fact, only has a usable Excel).

On the top, two terminal servers used to house the various clients. Why? Because The Man only installs and support IE6 and Excel 2003 in our PC environment. Nothing more. So I basically need to set up VMs which have the clients we need: ProcessBook, Excel 2010, and IE8 with the Adobe SVG player. One replicates the system which will host the clients used by our Joe Average User, which is web-only, and more worthy "Power Users", which can use Processbook if they like. Developers have their own environment with Excel 2010 from which they can publish directly on SharePoint. Hint: our production will go along the same way, using Citrix to deport the required applications. Screw The Man.

That's what is on my radar for now. I haven't finished making all these things work but when I'm done, I'll update and correct this post.

O.

My build of iperf is back

***********************************************************
Update: You can get my build of iperf on Windows here:
http://www.mayoxide.com/iperf
***********************************************************


Wew! My post concerning my own build of iperf for Windows has proven to be the most popular of my blog up to now. And the file got downloaded so many times, it actually busted the download limit of mayoxide.com (my domain) 12 hours before me leaving on vacation overseas. I had to renew my hosting package in a hurry, as I was keeping it grandfathered to 2004 transfer quotas - 1 gigabyte, anyone? Now I have 10 times more and the download link is back (thanks to pjmco.ca, my hosting provider, pay them a visit; they're great).

Such a response shows me that there is a need for a more "official" page for the Windows build of iperf, which is a matter of 30 minutes of handwritten HTML. I'll be working on this soon, I promise. In the mean time, my earlier blog post is a nice placeholder.

O.