Comments on Recording Screen Activity During Usability Testing

Karl Fast — Tue, 28 Aug 2007 00:42:22 GMT

Claudia:

The “special video driver” is installed as part of the ScreenCam installation process. You don’t need to download anything extra. If you’re already running ScreenCam this driver is already installed.

As for ScreenCam and XP, Lotus says on their site that ScreenCam (a) does NOT support Windows 2000 or XP and (b) it “has no plans to create a version of ScreenCam to work on Windows 2000 or Windows XP” (I quoted this on page 2 of the article). This was my major complaint about ScreenCam and why I didn’t recommend it.

Perhaps you meant Windows 98? ScreenCam works on Win95 and WinNT, and *should* work on Win98 depending on the video card. That’s according to this note on the ScreenCam site:

“The Windows 95 release of the ScreenCam recorder does work on Windows 98 (and its variants) with certain limitations due to some video cards driver/chip set manufacturers that do not completely comply with Microsoft recommendations and standards. The known video card drivers/chip set manufacturers that do not work with ScreenCam are detailed in technical support document #178038.”

http://www.lotus.com/products/screencam.nsf

I’m not sure where technical support document #178038 is, but if you visit the ScreenCam site I’m sure you can find it in the support documents.

Claudia Hurtado — Tue, 28 Aug 2007 00:42:22 GMT

I find your article very helpful since we are a company that has been using ScreenCam for quite a while years.
I got a question.. On page 2 you mention that screencam “Requires special video driver (easy to install).” can you explain? Also where can I find information about which video card will help me work with Windows XP and screencam?
Thanks for the help

Morgan Roberson — Tue, 28 Aug 2007 00:42:21 GMT

I very much appreciate your review of the screen recordingWe are a small company that wants to use screen recording demos of our products on the marketing cds we make and I was starting with absolutely no knowledge.
The review was very helpful in not only knowing what to test for but also how to test.
Thank you.

Morgan Roberson — Tue, 28 Aug 2007 00:42:21 GMT

Karl Fast — Tue, 28 Aug 2007 00:42:21 GMT

Several people have pointed out an error in the Camtasia section. In “The Bottom Line” it says that Camtasia does not run on Windows 2000 or XP, yet at the start of the page and in the Pros section it says it runs on all versions of Windows except Win95. Oops! It’s the latter that’s correct.

Camtasia runs on just about everything (so does Hypercam). It’s Lotus Screencam that doesn’t suppport Windows 2000/XP.

As for “FrontCam” at http://www.maxfeel.com/, I was previously unaware of the product. If I had heard of it before I might have included it in the review.

—karl

mandris — Tue, 28 Aug 2007 00:42:21 GMT

why not try frontcam at
http://www.maxfeel.com/
it can capture screen to both pictures and movies
and cheaper than camtasia

Troy Stein — Tue, 28 Aug 2007 00:42:21 GMT

Karl,

thanks for the review! Just two comments:

Camtasia supports Windows 2000 and XP. We’ve supported each OS within months of their respective releases.

On page four you talk about using HyperCam and TSCC. Its a cool trick. As an FYI, you need Camtasia installed on the same machine as Hypercam to make it work. The TSCC alone won’t do it. You likely had both installed and that’s why it was successful.

Thanks again for your review and recommendations! I hope you enjoyed your backpacking!

Troy Stein
Camtasia Marketing Manager

Tricia Tenpenny — Tue, 28 Aug 2007 00:42:21 GMT

Regarding the “Bottom Line” on the third page: It now says “Camtasia” but the content is still incorrect. Camtasia DOES run on Windows 2000 and XP.

Hey, nobody’s perfect!

Ron Zeno — Tue, 28 Aug 2007 00:42:20 GMT

Thanks very much, Karl, for an excellent clarification of your article!

Notes on scan conversion vs screen capture: Scan conversion to video (even S-VHS video) results in a poorer-quality image. It’s good enough for highlights, but bad for capturing 640×480 for analysis, and very bad for 800×600 or higher. Screen-capture software provides far better still image quality (lossless is possible), but the capture of movement will usually be poorer due to the slower frame rate.

George Olsen — Tue, 28 Aug 2007 00:42:20 GMT

Note: We’ve fixed the aforementioned typo on the third page.

Karl Fast — Tue, 28 Aug 2007 00:42:20 GMT

I wanted to respond to the comments about my article (note that I’m backpacking for four weeks and today is my resupply day, so this is the only response you’ll get from me until mid-September)

First, I *strongly* recommend testing Camtasia and Hypercam. It doesn’t take long and you can judge for yourself. (I do not recommend taking the time to test Screencam).

Secondly, here are my thoughts in response to specific comments from above:

** COMMENT: You have not provided enough information for me to decide if these tools have improved enough in the past two years for them to be of real use in usability testing.

MY RESPONSE: If you mean performance, then I would say yes, but this is mostly due to an improvement in hardware. If you’re talking about the software features useful specifically for usability people, then no. Like I said in the article these tools will not track keystrokes and other data to a log file so doing quantitative analysis is not possible. But in my experience you’re mainly looking for qualitative data here anyway (not always, but often).

There is certainly areas where these tools could improve. I suspect that you could use the Camtasia SDK to develop your own tool that would have thse features, but I have not investigated this.

** COMMENT: You failed to mention what purpose the tool is of acceptable use. Instead, you give a highly technical comparison that doesn’t map to any purpose that a usability specialist might want.

MY RESPONSE: Yes, I did give a technical comparison because in my experience the first problem with these tools has been technical. Getting good, high resolution video at a good frame rate with no performance degradation for the user is the first hurdle. These tools have finally cleared that hurdle. Now they need to work on features for usability testing.

BUT these features will probably not appear unless usability people ask for them. These tools were designed for people making training videos, not usability testing. I think usability folks should ASK these companies for the features they want and let them know that a large number percentage of their user base is using their products for a purpose they never thought of (read their websites and you’ll see that nobody lists “usability testing” as one of the uses for their product).

** COMMENT: What is the level of quality?

MY RESPONSE: Perfect. Each frame in the video could serve as a screenshot. No loss at all. Cursor visible throughout.

Main limitations here are (a) frame rate and (b) number of key frames captured (I did not discuss key frames in my article). The faster the machine the higher these numbers will be at a given resolution/color-depth. But as noted in my article, even a low-end machine by today’s standards will handle 1280×1024 at 16-bit color and 15 frames per second with IE at full screen and no performance impact for the user (my test machine was under $800 CDN, plus monitor and keyboard).

** COMMENT: Is this enough to give an overview of what happened in a usability test? Yes, from my experience. Enough to show exactly what a test participant did at any point in the test? (Not from my experience. Cannot always tell where the cursor is if the participant is working rapidly.)

MY REPONSE: Yes, if working rapdily this is a problem. Again, higher frame rates and more key frames solve this problem. Especially key frames, but this is not something I tested much.

However, this depends how quantitative your test is. If you need lots of quantitative data then these tools are unacceptable. But the vast majority of usability testing is done quickly and cheaply, guerrila-style, and that is almost entirely observational and qualitative.

Most usability tests will benefit from these tools. They are cheap, easy to use, do not cause problem for the user, and give you an visual recording that is clear, sharp, easy to read, and more than adequate for qualitative tests.

** COMMENT: What can fit on a CD-ROM for distribution? An hour or two of video?

MY RESPONSE: I did mention how big some of my test files were. Example, Camtasia produced a 36.8MB file for a 19.5 minute test in 800×600 at 16-bit color. Extrapolating you should get an hour test at this resolution/bit-depth at under 125 MB. So yes, an hour or two of video should easily fit on a CD.

As another person noted above, a 30 min video from Camtasia produces a 100MB file for them.

The factors are (a) screen resolution, (b) color-depth, (c) frame rate, (d) codec used, and (e) number of key frames. Your mileage will vary.

** COMMENT: Noticed on your 3rd page that your “Bottom Line” was from the Screencam product, not Camtasia.

MY RESPONSE: Yes, this is a typo. It should read “Camtasia.” Even my wonderful editors missed this.

Bryan — Fri, 26 Jan 2007 14:50:32 GMT

Karl:

Read your article with interest. Noticed on your 3rd page that your “Bottom Line” was from the Screencam product, not Camtasia.

Haven’t looked at these tools in a while, may do so again thanks to your review.

anu — Fri, 26 Jan 2007 14:50:32 GMT

We use camtasia as part of our usability testing toolset. Whether we use it on its own, or in conjunction with our mobile usability lab, we find it very useful. Screen cap on a [relatively] powerful laptop give enough fidelity to tell what a user did at any point in the test, and the abilty to capture sound alongside this is great, especially when we’re not taking the rest of our kit – asking the user to think aloud or make vocal notes during the session goes a long way to capturing essential info that you don’t get just from doing screen capture, and allows for after session review and analysis. I do agree that observer notes are as vital though.

A 30 minute session generates an .avi file of less than 100Mb, often a lot less.

Hope this helps

Jess — Tue, 28 Aug 2007 00:42:20 GMT

>to be of real use in usability testing

Well, Karl did say he was looking for 10 frames/sec. and he used a screencam for his “thinkafter” project. So that seems to be “real” usability testing. But the key consideration here is *cost*. Does a screencam tool provide value to low-budget usability evaluations? It can, but no guarantees (good notes from good observers go a long ways, without any fancy equipement. Video may not be needed at all, whether screencammed, camcordered, or scan converted.)

People looking to record screens who have a bit more money might also consider a consumer level scan converter to record computer monitor >> VCR.

Ron Zeno — Tue, 28 Aug 2007 00:42:20 GMT

Thanks Karl for some good information on screen-capture software. However, you have not provided enough information for me to decide if these tools have improved enough in the past two years for them to be of real use in usability testing. You failed to mention what purpose the tool is of acceptable use. Instead, you give a highly technical comparison that doesn’t map to any purpose that a usability specialist might want.

What is the level of quality? Enough to give an overview of what happened in a usability test? (Yes, from my experience.) Enough to show exactly what a test participant did at any point in the test? (Not from my experience. Cannot always tell where the cursor is if the participant is working rapidly.)

What can fit on a CD-ROM for distribution? An hour or two of video?