GpgOL Test Plan

Status: in progress

Introduction

The purpose of this test plan is to ensure GpgOL works well on all supported Microsoft Windows operating systems with all supported versions of Microsoft Outlook.

Test environment

Choose one combination of the following target platforms supported by GpgOL.

Microsoft Outlook:

• 2003 (32bit)
• 2007 (32bit)
• 2010 (32 or 64bit)
• 2013 (32 or 64bit)
• 2016 (32 or 64bit)

Operation system:

• Windows Vista (32 or 64bit)
• Windows 7 (32 or 64bit)
• Windows 8 (32 or 64bit)
• Windows 10 (32 or 64bit)

For interoperability tests additional mail clients will be tested together with Outlook/GpgOL:

• Thunderbird with Enigmail-Add-On (on Windows, MacOS or GNU/Linux)
• Apple Mail with GPGTools (on MacOS)
• Kontact Mail (KMail) enterprise 35 (on GNU/Linux)
• Kontact Mail (KMail) of KDE 4.14 (on GNU/Linux)

It is recommended to use a clean virtual test environment, e.g. with VirtualBox.

Preconditions

• email account A on IMAP/SMTP server
• email account B on Exchange Server 2013 or later
• OpenPGP public key (*.asc or *.gpg) and private key (*.asc or *.gpg)
• S/MIME certificate chain (*.pem) and private key (*.p12)

Tests

Testsuite 1: GpgOL basic crypto tests

Send crypto mails from/to same Outlook account.

1.1 PGP/MIME

Send mails with PGP/MIME:
a) sign and verify
b) encrypt and decrypt
c) sign/encrypt (with attachment*) and decrypt/verify (and check attachment)

1.2 S/MIME

Send mails with S/MIME:
a) sign and verify
b) encrypt and decrypt
c) sign/encrypt (with attachment*) and decrypt/verify (and check attachment)

(*) Attachment should contain umlauts in filename (e.g. fooäüö.txt)

Testsuite 2: GpgOL extended tests

2.1 Exchange Server

Run all tests from testsuite 1 via Exchange Server.

2.2 Move mails

Open an encrypted mail and move it from the inbox into an other folder. Check with an other IMAP client that all mails are always encrypted.

2.3 Drafts

Create new message. Mark message with "sign" and "encrypt". Close and save mail as draft. Open draft again. Check if the message is still marked with "sign" and "encrypt".

2.4 Cancel crypto operation

Create a new message. Mark as sign/encrypt. Click send. Cancel certificate selection dialog. Check if the message is not sent.

2.5 Sent messages

Send a signed/encrypted message. Check the sent folder ("Sent messages") if the mail is still signed/encrypted.

2.6 Reply

Reply to an encrypted mail. Check if the quoted text contains the decrypted message.

2.7 Disable GpgOL

Open an encrypted mail. Disable the GpgOL plugin in Outlook (via COM add-ons). Check that the opened mail is not saved decrypted. Select a OpenPGP signed/encrypted mail. Check that the mail cannot decrypt/verify.

2.8 Deactivate GpgOL's S/MIME (GpgSM)

Open a a S/MIME sign/encrypted mail (which was signed/encrypted with GpgOL). Deactivate S/MIME in GpgOL's option dialog. Close and reopen the mail two times. Check that Outlook's S/MIME implementation can decrypt/verify this mail at least the second time.

2.9 Close Outlook

Open a signed/encrypted mail. Close Outlook. Answer "Yes" if Outlook wants to save the changes. Restart Outlook.

Check with an other mail client (via IMAP) before and after restart of Outlook that the mail is never unencrypted on server.

2.10 Unknown recipient

Try to encrypt a message to a unknown email address. Check that Kleopatra cannot match a suitable certificate.

2.11 Cannot decrypt

Send an encrypted email to yourself but select an other certificate for encryption (not the own certificate). Check that GpgOL cannot decrypt the received message and the error indicates why.

2.12 Cannot sign

Remove your certificate (incl. secret key). Try to sign a new message. Check that GpgOL cannot sign the message and the error indicates why.

2.13 Cannot encrypt

Remove your certificate (if still available). Try to encrypt a new message to yourself. Check that GpgOL cannot encrypt the message and the error indicates why.

2.14 Signature with wrong address

Create or import new secret key which does not match to Outlooks email address. Send a signed mail to yourself but select the new certificate for signing. Verify the signature. Check the warning that the email address not contained in the certificate.

2.15 Attachment name with < 30% 8Bit encoding chars

Sign a Mail (not encrypt) and attach an attachment with the filename "tästfilename.txt". Check that signature is valid.

2.16 Attachment name with < 30% Utf-16 characters

Sign a Mail (not encrypt) and attach an attachment with the filename "test☃man.txt". Check that signature is valid.

2.17 Attachment name with mostly Utf-16 characters

Sign a Mail (not encrypt) and attach an attachment with the filename "☃". Check that signature is valid.

2.18 Forward / Reply to encrypted mail

Select forward or reply to an encrypted and signed mail. Is the plaintext inserted correctly? Is encrypt / sign preset to the state of the mail?

2.19 Office elements

Select Rich text mail and insert a Smart Art through the insert menu. Sign & Encrypt. Is an informative error shown?

2.20 Inline editors

In Outlook 2013 or 2016 reply to an encrypted mail in the messagelist. Are Crypto buttons shown? Does it work?

2.21 Closing Outlook

Open an encrypted mail. Close Outlook. When asked to save changes select yes. Verify with a different imap client that no plaintext is on the server.

2.22 Address book recipient

Add a Recipient to your Addressbook and select encrypt. Is the address correctly shown in certificate selection dialog?

2.23 Shared folders

Configure your IMAP or Exchange account to have read access to a folder of another user. Put encrypted messages in there with the other user. Decrypt. Does it work?

2.24 Kleopatra startup

Decrypt a mail. After decryption shut down Kleopatra. Select another Mail. Is Kleopatra restartet automatically?

Testsuite 3: Interoperability tests (Outlook's built-in S/MIME)

3.1 Receive S/MIME mails with GpgOL

Send S/MIME crypto mails from Outlook (with it's built-in S/MIME implementation) to an other Outlook (with GpgOL).

a) sign and verify
b) encrypt and decrypt
c) sign/encrypt (with attachment*) and decrypt/verify (and check attachment)

3.1 Send S/MIME mails with GpgOL

Send S/MIME crypto mails from Outlook (with GpgOL) to an other Outlook (with it's built-in S/MIME implementation).

a) sign and verify
b) encrypt and decrypt
c) sign/encrypt (with attachment*) and decrypt/verify (and check attachment)

Testsuite 4: Interoperability tests (Kontact Mail)

Send crypto mails between Kontact Mail (KMail) and Outlook (with GpgOL).

4.1 PGP/MIME

a) sign (KMail) -> verify (GpgOL)
b) encrypt (KMail) -> decrypt (GpgOL)
c) sign/encrypt with attachment (KMail) -> decrypt/verify and check attachment (GpgOL)
d) answer to (a): sign (GpgOL) -> verify (KMail)
e) answer to (b): encrypt (GpgOL) -> decrypt (KMail)
f) answer to (c): sign/encrypt with attachment (GpgOL) -> decrypt/verify and check attachment (KMail)

4.2 S/MIME

a) sign (KMail) -> verify (GpgOL)
b) encrypt (KMail) -> decrypt (GpgOL)
c) sign/encrypt with attachment (KMail) -> decrypt/verify and check attachment (GpgOL)
d) answer to (a): sign (GpgOL) -> verify (KMail)
e) answer to (b): encrypt (GpgOL) -> decrypt (KMail)
f) answer to (c): sign/encrypt with attachment (GpgOL) -> decrypt/verify and check attachment (KMail)

4.3 INLINE/PGP

a) sign (KMail) -> verify (GpgOL)
b) encrypt (KMail) -> decrypt (GpgOL)
c) sign/encrypt without attachment (KMail) -> decrypt/verify (GpgOL)

Testsuite 5: Interoperability tests (Thunderbird)

Send crypto mails between Thunderbird (with Enigmail) and Outlook (with GpgOL).

5.1 PGP/MIME

a) sign (TB) -> verify (GpgOL)
b) encrypt (TB) -> decrypt (GpgOL)
c) sign/encrypt with attachment (TB) -> decrypt/verify and check attachment (GpgOL)
d) answer to (a): sign (GpgOL) -> verify (TB)
e) answer to (b): encrypt (GpgOL) -> decrypt (TB)
f) answer to (c): sign/encrypt with attachment (GpgOL) -> decrypt/verify and check attachment (TB)

5.2 S/MIME (with Thunderbird's S/MIME implementation)

a) sign (TB) -> verify (GpgOL)
b) encrypt (TB) -> decrypt (GpgOL)
c) sign/encrypt with attachment (TB) -> decrypt/verify and check attachment (GpgOL)
d) answer to (a): sign (GpgOL) -> verify (TB)
e) answer to (b): encrypt (GpgOL) -> decrypt (TB)
f) answer to (c): sign/encrypt with attachment (GpgOL) -> decrypt/verify and check attachment (TB)

5.3 INLINE/PGP

a) sign (TB) -> verify (GpgOL)
b) encrypt (TB) -> decrypt (GpgOL)
c) sign/encrypt with attachment (TB) -> decrypt/verify and check attachment (GpgOL)

5.4 PGP/Partioned

a) sign (TB) -> verify (GpgOL)
b) encrypt (TB) -> decrypt (GpgOL)
c) sign/encrypt with attachment (TB) -> decrypt/verify and check that you have the option to decrypt attachment (GpgOL)

Testsuite 6: Interoperability tests (GPGTools)

Send crypto mails between Apple Mail (with GPG for Mail plugin from GPGTools) and Outlook (with GpgOL).

6.1 PGP/MIME

a) sign (Apple Mail) -> verify (GpgOL)
b) encrypt (Apple Mail) -> decrypt (GpgOL)
c) sign/encrypt with attachment (Apple Mail) -> decrypt/verify and check attachment (GpgOL)
d) answer to (a): sign (GpgOL) -> verify (Apple Mail)
e) answer to (b): encrypt (GpgOL) -> decrypt (Apple Mail)
f) answer to (c): sign/encrypt with attachment (GpgOL) -> decrypt/verify and check attachment (Apple Mail)

6.2 S/MIME

a) sign (Apple Mail) -> verify (GpgOL)
b) encrypt (Apple Mail) -> decrypt (GpgOL)
c) sign/encrypt with attachment (Apple Mail) -> decrypt/verify and check attachment (GpgOL)
d) answer to (a): sign (GpgOL) -> verify (Apple Mail)
e) answer to (b): encrypt (GpgOL) -> decrypt (Apple Mail)
f) answer to (c): sign/encrypt with attachment (GpgOL) -> decrypt/verify and check attachment (Apple Mail)