justinclarke.com

I use PGP on most days to send encrypted and digitally signed email to external clients. However, an increasing number of my clients are using the S/MIME standard to encrypt external email instead.

Since my company uses Lotus Notes, I went through the process of figuring out how to get Lotus Notes to send S/MIME encrypted or signed email (supported in recent versions - these screenshots are from Notes 6.5.3).

First of all, you will need to get an X.509 email certificate. You can get these for free from Thawte, or you can pay to get them from Verisign.

The example I am going to use is Thawte. For this example, I am assuming you have already requested the certificate using Internet Explorer, and you have already downloaded the certificate into IE. The guidance on the Thawte site should be able to get you to that point.

Firstly, for those not familiar with exporting certificates, you need to go to Tools -> Internet Options -> Content, and click on the Certificates button in the middle :-

You should then be presented with a screen as below where you can export the certificates installed. If you don't see a certificate issued by Thawte at this point, you haven't got the certificate installed in IE correctly.

At this point we click "Export" to start the export wizard - make sure that we export the private key, and use PKCS #12 format (a .pfx file). You will be required to pick a password - this is used to provide some security over the key, especially useful as we have the private key in the file.

At this point we can import the key into Lotus Notes. This is done using the File -> Security -> User Security menu. In the "Your Identity" section there is a subsection titled "Your Certificates". If you click on this subsection, you should get a screen similar to this one :-

If you click on "Get Certificates", and select "Import Internet Certificates", you should be prompted to select the PKCS #12 file created when the certificates were exported from IE. Select that file, enter the password you specified before, and you should see a screen similar to this :-

Click "Accept All", and you should be finished.

Note that in order to sign or encrypt email you will need to specify this in the delivery options for each message, or change your user options to automatically sign/encrypt all messages.

In order to import X509 certificates from people who have sent you signed messages, you will need to add them to your address book using Tools -> Add Sender to Address Book, making sure that on the Advanced tab, "Include X509 certificates when encountered" is checked.

I've added a Google Toolbar button for my blog - if you're interested click this link to add justinclarke.com to your Google Toolbar. You'll need the latest beta version of the toolbar for this to work.

I note that Nitesh has some details about a service available in the US to allow you to do Caller ID spoofing very easilly using a calling card. This is somewhat easier than having your own PABX and programming the outbound number :-)

Who knew it would be so hard to find details on how to turn the TRACE method _on_ for an IIS 6.0 box? In order to test a plugin for Oedipus I needed to turn the TRACE method back on for my testing server running Windows 2003 and IIS 6.0. I finally found the required detail here at the Microsoft site.

For reference - TRACE is controlled by a registry key. Create a DWORD value called EnableTraceMethod in HKEY_LOCAL_MACHINE\ SYSTEM\ CurrentControlSet\ Services\ W3SVC\ Parameters. This should be 1 for on, and 0 for off. Enjoy!

I recently had the need to generate some flash demos to demonstrate the Oedipus Web Scanner in action. Since I don't happen to have one of the commercial Flash demo generators handy at home (I use Viewlet Builder for Linux at work) I decided to see what I could get done with an all Open Source solution.

I quickly found vnc2swf, for recording a VNC session as a Flash movie. There are two versions of vnc2swf, one in C and one in Python - I ended up using both in my case. A cool feature both version have is they generate an HTML file that you can use to display the Flash file you just created. The Python version, which is the one under active development, generates a nice Javascript progress bar and pause button in this page, which I ended up using verbatim for my demo pages.

Since I was recording on Linux I installed TightVNC as my VNC server (you can use any VNC server, such as RealVNC or UltraVNC, or the original VNC if you still use it).

Running vncserver on Linux starts a separate X Windows session that can have it's own programs running. This was perfect for my needs, however if you need access to your entire desktop for some reason you could use x11vnc for this. For the purposes of what I needed, I configured my VNC session to run gnome terminal and metacity so I could get a Gnome-style capture. You do this by configuring your $HOME/.vnc/xstartup file on the server I was using. This is what mine looks like:

#!/bin/sh
xrdb $HOME/.Xresources
xsetroot -solid grey
gnome-terminal &
metacity &

After this is all setup, we point vnc2swf at the vncserver. In my case, the options I used (using the C version of vnc2swf) were similar to this:

vnc2swf -startrecording -geometry 800x600 -depth 16 -framerate 5 ~/moviename.swf hostname:1.0

After recording a satisfactory demo, I recorded an audio commentary as a 44kbps WAV file, compressed this to a 24kbps MP3 file using LAME, and added this to the original Flash file (and did some compression as well) using edit.py from the Python version of vnc2swf as follows:

edit.py -a audio.mp3 -c -o output.swf original.swf

And we're done! I should note that one of my audio files was a few seconds too long. I used Audacity to "speed up" the file so that it fit the length of the flash movie.

For reference, the demos I recorded are here and here to see what kind of quality I ended up with.

Final Note: A colleague pointed me to Wink after I had completed this. I will be trying this next time I need to do a software demo.

From reviewing the logs on my website, a lot of the hits I get are from various RSS feed readers and aggregators, especially of the Google and Yahoo variety. Since I don't blog that often (as this is really a forum for tips, tricks, howtos, and articles), I thought I might find out how to specify to an RSS feed reader that my blog is not updated that often.

I eventually found this site, which had the detail I was looking for RSS 1.0 and RSS 2.0 feeds. You should now all see that my feed "suggests" that the update internal is 1440 minutes (one day) between updates.

Now if only there were some support for the same in the Atom format?

Recently I decided for some reason to run my site through the W3C Validator to check to see if I was generating valid XHTML. If you've ever done this, you shouldn't be surprised to find out that my site was anything but compliant. Why is this important? Well, it's probably not going to break most modern browsers as they are quite tolerant of non-compliant code, but it's bad coding - if your code is valid then you should be displayed well by any compliant browser or parser in use.

Some of the things that I wasn't doing correctly included:

• Terminating break and image tags:
  • <br> - Incorrect
  • <br /> - Correct
• Handling Movable Type's default "Convert Line Breaks" behaviour that sticks tags in lists where they aren't welcome
• Including "alt" elements in my IMG tags
• Making all of my tags lower case. In XHTML, HREF and href are different - lower case is correct
• Encoding all ampersands (&), including ones in URLs

And, my number one problem - multl layer lists:

<ul>
<li>This is incorrect</li>
<ul>
<li>This should be indented</li>
</ul>
</ul>

Which should actually be written like this:

<ul>
<li>This is correct
<ul>
<li>This will be indented</li>
</ul>
</li>
</ul>

I enlisted a few tools in my quest to get valid and stay valid. Foremost was the W3C Validator service, which I have integrated into my publishing routine using Brandon Fuller's excellent MT-W3CValidator plugin, and the handy XHTML reference material at W3Schools. I am proudly displaying my XHTML valid status on each page (as verified by the W3CValidator plugin), so feel free to click the icons to verify the validity of my XHTML, Atom, RSS 1.0, and/or RSS 2.0 feeds.

This is going to be installment one of quite a few little tricks and tips that I never seem to remember, and consequently alsways end up having to go out an research again. Hence, I'll now be blogging these little gems so they are in one place.

Sed is very useful for editing a stream (i.e. a file) on the fly. The most common thing I use it for is in a script in order to replace a setting in a file.

sed -e '/foo/s//bar/g' - replace all instances of foo (a regex) with bar. Note that the -e is not needed if there is only one "script" to process.

Update - Thanks to mat (see comments) for pointing out my fat fingering - sed -e '/foo/s/bar//g'

Another one I always use - doing some command over every line of a file:

for i in `cat foo.txt`; do echo $i; done

Nice and simple, but works a treat.

I was looking for the easiest way to migrate my email off my aging iBook G3 and onto my new.... gasp.... Dell Inspiron 6400 running Office 2003. After searching Google to see if there were any free or open source solutions that I would trust my data to, I came across a posting that mentioned using IMAP (Internet Message Application Protocol) to do this.

For those of us who don't use IMAP on a regular basis, its a widely supported email protocol that involves leaving your email on the server. Since your email remains on the ISP's server, you can read, manipulate and send email from multiple computers while maintaining all of that activity centrally. This differs from the more common POP (Post Office Protocol) which downloads the email off the ISP's server onto your local machine. Perhaps not so strangely, some ISPs either don't support, or don't advertise, that IMAP support is available. Maybe to save space on their email servers?

So how do you go about using an IMAP account to migrate your emails across? This is the process I went through on both Apple Mail and Entourage in order to migrate my email to the Windows PC running Outlook 2003. Its pretty simple. Note that you will probably end up with some duplicates this way if your POP mail and IMAP mail are the same inbox - I just deleted mine manually:

• I added my IMAP email account to Apple Mail/Entourage on the OS X machine. This was in addition to the existing POP account that I already had setup. These will normally be shown in the mail client interface as a separate set of folders to your normal email (in Apple Mail, a large "@" logo, in Entourage a folder looking like a letter attached to a network)
• For each Inbox and folder I had in Apple Mail and Entourage I created a folder in my IMAP account. For Apple Mail this involved creating a folder in the IMAP account through using Control-Click on the IMAP account, and selecting "New Mailbox". For Entourage through using Control-Click on the IMAP account, and selecting "New Folder"
• I then copied all of the messages in a folder to the matching folder on the IMAP server by selecting all of the messages in my email client (highlighting, or using Apple-A) and dragging the messages across. This process took quite a while for larger folders (i.e. 2000+ messages), and for Apple Mail you don't get any indication of progress except the spinning activity indicator, so you might want to check back after while to see if it is done. For Entourage, you get a progress indicator, and it even seems to work when you start multiple copies at the same time (YMMV)
• I then did what amounts to the reverse process in Outlook on the PC. I set up the IMAP account in Outlook, created folders in my Personal Folders for the email I wanted to categorise, and copied the emails from the IMAP server to the folders I had created

While this process took quite a while to copy the thousands of emails I have, it went fairly smoothly. For the Apple Mail to Outlook migration, all of the emails were copied across without a problem, preserving the dates and formatting. For some reason, the dates on the emails were not preserved during the Entourage to Outlook migration. Annoying? Yes. But hardly a showstopper for me since the email is largely for reference anyway. Enjoy!

It's always those little tools that you are looking for and can never find... in my case dos2unix for stripping a text file of all of the pesky ^M characters that get put in on the Windows platform. For reference, with Ubuntu, dos2unix and unix2dos are found in the tofrodos package.

Here is another useful snippet for bash - doing something a number of times. The for loop again:

for (( i=0; i<100; i++ )); do echo $i; done

Daniel Cuthbert commented about adding proxy server support to SQLBrute. Well, a nice (and perhaps a little obscure) feature of urllib2 (which SQLBrute uses to send HTTP requests) is that you get HTTP proxy support out of the box. All you need to do is set your environment to point to the server, either by setting the http_proxy environment variable to point to your proxy, or (on Windows) by setting Internet Explorer to point to your proxy.

If setting the environment variable, on the Windows command line you can do this:

set http_proxy=myproxy:8080

Which should point it to your proxy. The same works for Unix/Linux with whatever export or set command is relevant for your shell.

The proxy support in urllib2 doesn't support authentication so if you have a proxy with authentication you might want to chain SQLBrute through your favourite local proxy (i.e. Burp, Paros etc) and have that handle the authentication for you. Enjoy!

I recently had reason to spend a while working with Nessus on Windows XP (Service Pack 2). Usually, I use a Nessus Server running on Linux, either running locally if I am onsite, or one installed on our company infrastructure for scanning from the Internet. In fact, you read the documentation don't you?, Tenable specifically recommends in the Nessus Installation Guide that you _not_ run Nessus on XP, and instead use a Windows Server product, such as Windows Server 2003.

The reason for this is that in Windows XP Service Pack 2, Microsoft introduced a number of Network Protection Technologies for mitigating the spread of malware. One of these limits the number of simultaneous incomplete outbound TCP connection attempts to 10, with additional attempts being queued and potentially dropped. This impacts the reliability of at least port scanning, and possibly other security checks.

Unfortunately the scenario I was working with required me to be running Nessus through a VPN client (never ideal), in reality requiring me to be on XP. Tenable does, however, have some recommendations for running Nessus as reliably as possible on XP:

• Max number of hosts: 10
• Max number of security checks: 4
• Max number of packets per second for port scan: 50

The maximum hosts/security checks setting is standard in all of the Nessus clients I've used, however the packets per second setting seems to only be available within the client shipped with the Windows Nessus server. If you, like me, are using the new NessusClient 3.0 beta for Windows, you need to make the following change to the Nessus server's configuration to ensure that 50 is the default value:

• Go to the "config" directory in your Nessus server installation. By default this is C:\Program Files\Tenable\Nessus\config
• Open config.default.xml for editing - just use Notepad if you don't have an XML editor
• Find the SYN Scan:Max number of packets per second for port scan node, and edit the value (the CDATA bit) from 500 to 50

This value should now be the default for all new scans.

This worked well for me, however needless to say that running a Nessus scan in VMWare (slowdown factor one), over a VPN link (slowdown factor two), over a transatlantic Internet connection (slowdown factor three), the scan took quite a while to complete...