Posts

Showing posts from 2014

Don't let Java on Linux determine its own Timezone

Abstract: In this post, I illustrate one reason why its useful not to let Java (at least on Linux) determine its own timezone. This ends up being particularly important when considering inter-version compatibility issues. This post is not specific to any product, but does use Oracle WebCenter as a real-life case-study.

I've been diagnosing a fault one some of our Java environments with Oracle WebCenter 11gR1 on RHEL6. The nature of the fault is such that we don't know when it started occurring, the machine (and the feature involved) is not in routine use.

In Web Center, you can transfer data between environments using an 'Archiver' (which at least in earlier versions is also a ... topic for some other conversation). As part of setting that up, you need to create a 'Provider' (a way of getting information in or out of the system). Each provider has a useful 'Test' button.... or at least, it has a 'Test' button, and when you click it, it should sa…

Building an RPM: debuginfo package not being created

(Very short version, for the benefit of the frustrated: comments in RPM SPEC files don't do what you think).

I was building an RPM for NXLOG (http://nxlog-ce.sourceforge.net/) because I needed to get a debuginfo package in order to help diagnose a memory leak from the vendor's supplied SPEC file. NXLOG helpfully includes a make_rpm.sh script in its source-code. What I expected to see near the end of the build was the following output:

...
Wrote: /home/cameron/src/rpm/SRPMS/nxlog-ce-2.8.1248-1.src.rpm
Wrote: /home/cameron/src/rpm/RPMS/x86_64/nxlog-ce-2.8.1248-1.x86_64.rpm
Wrote: /home/cameron/src/rpm/RPMS/x86_64/nxlog-ce-debuginfo-2.8.1248-1.x86_64.rpm
...
However, it wasn't building the debuginfo package (that line was missing).

Performance Analysis of Java Middleware on Linux

I routinely have to look after some reasonably complex Java middleware deployments, deployed variously on container technology provided by Tomcat or Oracle WebLogic. The hardest, and generally the most useful, thing to determine is identifying which resource is being constrained (commonly not CPU or OS memory, but often things like number of threads [dedicated to something like database connection pool]).

This is a post that I intend on maintaining as I document (and discover, hopefully) more tips and tricks; because sometimes its just not so great being the Go To Guy when it comes to engaging your head against a brick wall.

(Ab)using Samba and inotify to implement simple menu of privilegedactions [Part 3: Basic Implementation]

Okay, so I got it working; but more as the first-generation system that I sketched out in my design notes. Ie. one trigger maps to one action, and there is no separation between objects and actions. When I set it up and gave it to my client to try out, she send me a text message with some feedback; it said "That's cool!", I'm happy. I dare say this will get a (little) more polished in subsequent deployments; it would be good to separate the configuration from the application logic.

Here is what it looks like in action; note that this is done over CIFS, so the reactivity of the interface will depend on whether Samba on the server, and you CIFS client, handles update notifications. For example, on my aging RHEL 6 GNOME 2 desktop, it does not (I have to hit refresh repeatedly); but I gather from my client's Mac, it does. You can see what it looks like from Windows from this tiny screencast I made:



Capturing and Analysing DNS Samples (tcpdump meets SQLite3)

A few nights ago, I was on a bit of a mission to determine how much traffic our servers were sending to our DNS servers, as I was fairly sure it was much, and I wanted to put some improvement in place so we don't make an issue for ourselves.

I didn't want to turn on query logging, because of the rate of traffic, and its not my job to manage the DNS service, but I can see the queries in flight, so tcpdump was an obvious tool to use. But I didn't want to be awking, grepping, sedding, and generally bashing (pun intended) tcpdump output into whatever variety of reports I wanted, so I instead fed it into SQLite3.

(Ab)using Samba and inotify to implement simple menu of privileged actions [Part 2: Proof of Concept Implementation]

In my last post, I ran through the design; this post is shows the result of my initial proof-of-concept. It was interesting to play with some new modules in Python I hadn't previously used, including Python's threading.Timer and of course pyinotify, and the subprocess and shlex modules, which I'm already familiar with, but it rates a mention.

Here's the code. Needless to say, it needs some tidying up, but I think the basic principles and threading correctness seem okay.

(Ab)using Samba and inotify to implement simple menu of privileged actions [Part 1]

Part 1: Design Analysis

I expect there will be at least one other part that covers the implementation, and another part covering how to use it.

Let's say you offer some form of software as a service to customers, such as a website with a database, middle-ware and web tier. In order to limit exposure, you have a policy not to allow console access via tools such as SSH or RDP. You might instead offer access to various directories using tools such as Samba, and perhaps remote access to the database (over SSL) if required. Samba could also provide access to the logs.

Ah, but if someone has access to change something in a configuration, such as in the middleware layer or web tier, how then would they restart things? A few options come to mind. The first might be some restricted access via SSH where the user is forced into a menu-driven interface. Another might be some web-interface (such as cPanel). Those would be the obvious contenders, let's look at each before deciding if it i…

ORA-12170: TNS:Connect timeout — resolved

If you're dealing with Oracle clients, you may be familiar with the error message
ERROR ORA-12170: TNS:Connect timed out occurred I was recently asked to investigate such a problem where an application server was having trouble talking to a database server. This issue was blocking progress on a number of projects in our development environment, and our developers' agile post-it note progress note board had a red post-it saying 'Waiting for Cameron', so I thought I should promote it to the front of my rather long list of things I needed to do... it probably also helped that the problem domain was rather interesting to me, and so it ended being a late-night productivity session where I wasn't interrupted and my experimentation wouldn't disrupt others. I think my colleagues are still getting used to seeing email from me at the wee hours of the morning.

This can masquerade as a number of other error strings as well. Here's what you might see in the sqlnet.log f…

Getting MySQL server to run with SSL

Image
I needed to get an old version of MySQL server running with SSL. Thankfully, that support has been there for a long time, although on my previous try I found it rather frustrating and gave it over for some other job that needed doing.

If securing client connections to a database server is a non-negotiable requirement, I would suggest that MySQL is perhaps a poor-fit and other options, such as PostgreSQL -- according to common web-consensus and my interactions with developers would suggest -- should be first considered. While MySQL can do SSL connections, it does so in a rather poor way that leaves much to be desired.

UPDATED 2014-04-28 for MySQL 5.0 (on ancient Debian Etch).

Here is the fast guide to getting SSL on MySQL server. I'm doing this on a Debian 7 ("Wheezy") server. To complete things, I'll test connectivity from a 5.1 client as well as a reasonably up-to-date MySQL Workbench 5.2 CE, plus a Python 2.6 client; just to see what sort of pain awaits.

UPDATE: 2014-0…

Influencing Python's choice of SSL/TLS cipher-suite

I'm debugging a fault being received by an application that uses the Python [v2.7] SOAP library (module name 'suds'). I want to look inside the SSL datastream, much as I can between SoapUI and the same server. SoapUI, being the typical Java client, doesn't default to a particularly high grade of cryptography, and so with the private key I can first record and then later inspect the cleartext data with Wireshark.

But Python, like many others, seems to default to enabling a lot of more secure cipher-suites that enable perfect-forward-secrecy. In order to snoop on those, you need to get the client to dump out the pre-master key somewhere.This post shows how you can you get Python's SSL module to use a different cipher specification string.

Making Cluster-SSH (and regular SSH) a lot more usable with regard to reconnecting

Image
If you find yourself patching a lot of machines at once, and reboot, then your SSH window will close.... not very useful if you want to keep track of a number machines you need to log back into to check that all is okay, or to start services that don't start automatically. It makes that time of the month -- patching -- rather more tedious and painful than it ought to be.

Enter a useful tool called Cluster SSH (command name 'cssh', package name 'clusterssh', version used is 3.28 from EPEL). It distributes my keystrokes to all of the windows that it starts. You can toggle, add and remove hosts to manage, and you can configure clusters of machines. While it does lack polish, it is very useful in reducing the amount of time it takes to patch a lot of machines; I estimate that it takes the time required to about a third.

Here's an example of using it 'in anger' while patching 37 machines. I've deliberately made the image small enough so as to make any te…