Hey guys,
TestRail, for the most part, is a testing dream come true for me! But, one of the most important parts of testing is letting product managers and other business people know the results of test passes. Of course, this is where reporting comes in.
Unfortunately, the reporting built-in to TestRail, though comprehensive, is nigh unusable. The reason lies within the available methods of delivery. I'll go over where the problems lie:
- Email a link to the report.
1. If we're going to bother sending an e-mail about a test report, the e-mail should contain the content of the test report, rather than just a link to view the report in Test Rail. This would be a minor problem, but for:
2. In order to view a TestRail report, one must have a TestRail account. We don't want to use up TestRail licenses just so a VP can glance at a report once in a while. It would be great if there were a way to either make free, view-report-only accounts, or to make reports generally public.
- Email the report as an attachment.
The attachment, as you know, comes in the form of a zip archive of a web page. This presents several problems.
1. Security. Most people (hopefully!) are fairly guarded when they receive an e-mail with an attachment -- there's always the concern that "Your test result is attached" is social engineering. This could be overcome with some education of the intended audience, but the tool should accommodate the audience -- not vice-versa.
2. Anti-virus. My corporation's e-mail server strips the javascript files out of the attachments, since .js files in an e-mail could be malicious scripts. When opening the webpage, this makes the pie charts, among other things, disappear. We also get e-mails from the e-mail server that the .js files were stripped from that e-mail. This isn't a good experience for our intended audience, which could go as high as VPs.
3. Getting a non-technical business person to unzip a file and open index.html. As above, busy folks don't want to take the time to pursue a test report. They certainly don't want to find where the report unzipped to, and look through some obscure-sounding filenames for index.html.
Ultimately, we need reports in the body of e-mail messages. I understand that this is not currently done because the Outlook HTML rendering engine is based on Word, and the reports don't show up with the active content of the website reports, and also some goofy rendering. At a minimum, I'd like even a very simplified report, using basic tables and some nice fonts, where possible. The active content, while nice, is not necessary for viewing in an e-mail.
I will probably end up writing my own reporter to accomplish this, though I'd greatly prefer a turnkey solution.
Your work is otherwise amazing! This is a fantastic tool, and I pushed hard on execs up to the VP level to get its purchase approved for the company.