Differences between revisions 2 and 3

GpgOL test plan (DRAFT)

Here you can find information about the manual testing progress of GpgOL.

Contents

GpgOL test plan (DRAFT)

What is (going to be) tested

We want to ensure GpgOL works well on all supported Microsoft Windows operating systems with all supported versions of Microsoft Outlook.

The tests refer to the signing and encrypting process in Microsoft Outlook using GpgOL, the transmission of encrypted and (or) signed emails, and the receiving of encrypted and (or) signed emails.

The tests are not only performed with encrypted and signed messages from and to Microsoft Outlook, but also with other Windows and Linux Mail-Clients which are supported by Gpg.

Especially:

Microsoft Outlook 2010 (On Windows 7 64bit)
Microsoft Outlook 2013 (On Windows 8 64bit)
Microsoft Outlook 2016 (On Windows 7 32bit)
Microsoft Outlook 2016 (On Windows 8 64bit)
Microsoft Outlook 2016 (On Windows 10 64bit)
thunderbird (Windows and Linux)
KMail (Kontact e35 Linux)

The tests will refer to the usage of PGP/MIME and PGP/Inline.

About the testing-environment

To test out the different Versions of Microsoft Outlook on different platforms and CPU-Architectures a virtual test-Environment was created using Oracle VirtualBox.

Test plan

The test cases described in plain text

1. Send a Mail from a Outlook Client to an Outlook Client. Use OpenPGP/Mime for the signing process

for Microsoft Outlook 2010 (On Windows 7 64bit)
- a) Sign the E-Mail
- b) Encrypt the E-Mail
- c) Sign and Encrypt the E-Mail in one step
- d) Verify signature of the E-Mail you just sent
- e) Decrypt the E-Mail you just sent
- f) Verify and decypt the email you just sent in one step
for Microsoft Outlook 2013 (On Windows 8 64bit)
- a) Sign the E-Mail
- b) Encrypt the E-Mail
- c) Sign and Encrypt the E-Mail in one step
- d) Verify signature of the E-Mail you just sent
- e) Decrypt the E-Mail you just sent
- f) Verify and decypt the email you just sent in one step
for Microsoft Outlook 2016 (On Windows 7 32bit)
- a) Sign the E-Mail
- b) Encrypt the E-Mail
- c) Sign and Encrypt the E-Mail in one step
- d) Verify signature of the E-Mail you just sent
- e) Decrypt the E-Mail you just sent
- f) Verify and decypt the email you just sent in one step
for Microsoft Outlook 2016 (On Windows 8 64bit)
- a) Sign the E-Mail
- b) Encrypt the E-Mail
- c) Sign and Encrypt the E-Mail in one step
- d) Verify signature of the E-Mail you just sent
- e) Decrypt the E-Mail you just sent
- f) Verify and decypt the email you just sent in one step
for Microsoft Outlook 2016 (On Windows 10 64bit)
- a) Sign the E-Mail
- b) Encrypt the E-Mail
- c) Sign and Encrypt the E-Mail in one step
- d) Verify signature of the E-Mail you just sent
- e) Decrypt the E-Mail you just sent
- f) Verify and decypt the email you just sent in one step

2. Send a Mail from a Outlook Client to an Outlook Client. Use S/Mime for the signing process

for Microsoft Outlook 2010 (On Windows 7 64bit)
- a) Sign the E-Mail
- b) Encrypt the E-Mail
- c) Sign and Encrypt the E-Mail in one step
- d) Verify signature of the E-Mail you just sent
- e) Decrypt the E-Mail you just sent
- f) Verify and decypt the email you just sent in one step
for Microsoft Outlook 2013 (On Windows 8 64bit)
- a) Sign the E-Mail
- b) Encrypt the E-Mail
- c) Sign and Encrypt the E-Mail in one step
- d) Verify signature of the E-Mail you just sent
- e) Decrypt the E-Mail you just sent
- f) Verify and decypt the email you just sent in one step
for Microsoft Outlook 2016 (On Windows 7 32bit)
- a) Sign the E-Mail
- b) Encrypt the E-Mail
- c) Sign and Encrypt the E-Mail in one step
- d) Verify signature of the E-Mail you just sent
- e) Decrypt the E-Mail you just sent
- f) Verify and decypt the email you just sent in one step
for Microsoft Outlook 2016 (On Windows 8 64bit)
- a) Sign the E-Mail
- b) Encrypt the E-Mail
- c) Sign and Encrypt the E-Mail in one step
- d) Verify signature of the E-Mail you just sent
- e) Decrypt the E-Mail you just sent
- f) Verify and decypt the email you just sent in one step
for Microsoft Outlook 2016 (On Windows 10 64bit)
- a) Sign the E-Mail
- b) Encrypt the E-Mail
- c) Sign and Encrypt the E-Mail in one step
- d) Verify signature of the E-Mail you just sent
- e) Decrypt the E-Mail you just sent
- f) Verify and decypt the email you just sent in one step

3. Send an E-Mail with KMail to an Outlook Client. Check if the Outlook Client can perform the following tasks without errors

Perform the Steps using OpenPGP/Mime
1. Microsoft Outlook 2010 (On Windows 7 64bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step
2. Microsoft Outlook 2013 (On Windows 8 64bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step
3. Microsoft Outlook 2016 (On Windows 7 32bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step
4. Microsoft Outlook 2016 (On Windows 8 64bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step
5. Microsoft Outlook 2016 (On Windows 10 64bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step
Perform the Steps using S/Mime
1. Microsoft Outlook 2010 (On Windows 7 64bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step
2. Microsoft Outlook 2013 (On Windows 8 64bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step
3. Microsoft Outlook 2016 (On Windows 7 32bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step
4. Microsoft Outlook 2016 (On Windows 8 64bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step
5. Microsoft Outlook 2016 (On Windows 10 64bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step

4. Send an E-Mail with Thunderbird to an Outlook Client. Check if the Outlook Client can perform the following tasks without errors

Perform the Steps using OpenPGP/Mime
1. Microsoft Outlook 2010 (On Windows 7 64bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step
2. Microsoft Outlook 2013 (On Windows 8 64bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step
3. Microsoft Outlook 2016 (On Windows 7 32bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step
4. Microsoft Outlook 2016 (On Windows 8 64bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step
5. Microsoft Outlook 2016 (On Windows 10 64bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step
Perform the Steps using S/Mime
1. Microsoft Outlook 2010 (On Windows 7 64bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step
2. Microsoft Outlook 2013 (On Windows 8 64bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step
3. Microsoft Outlook 2016 (On Windows 7 32bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step
4. Microsoft Outlook 2016 (On Windows 8 64bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step
5. Microsoft Outlook 2016 (On Windows 10 64bit)
  - a) Verify the signature
  - b) Decrypt the mail
  - c) Verify and Decrypt the E-Mail in one step

Test plan matrix

1. Sender and receiver is Outlook on Windows using OpenPGP/Mime

o	1) OL2010 W7_64	2) OL2013 W8_64	3) OL2016 W7_32	4) OL2016 W8_64	5) OL2016 W10_64
a) Signing	?	?	?	?	?
b) Encrypting	?	?	?	?	?
c) Sig+Enc	?	?	?	?	?
d) Verify signature	?	?	?	?	?
e) Decrypt	?	?	?	?	?
f) Verify + Decrypt	?	?	?	?	?

2. Sender and receiver is Outlook on Windows using S/Mime

o	1) OL2010 W7_64	2) OL2013 W8_64	3) OL2016 W7_32	4) OL2016 W8_64	5) OL2016 W10_64
a) Signing	?	?	?	?	?
b) Encrypting	?	?	?	?	?
c) Sig+Enc	?	?	?	?	?
d) Verify signature	?	?	?	?	?
e) Decrypt	?	?	?	?	?
f) Verify + Decrypt	?	?	?	?	?

3. Sender is KMail receiver is Outlook

3.1 using OpenPGP/Mime

o	1) OL2010 W7_64	2) OL2013 W8_64	3) OL2016 W7_32	4) OL2016 W8_64	5) OL2016 W10_64
a) Verify signature	?	?	?	?	?
b) Decrypt	?	?	?	?	?
c) Verify + Decrypt	?	?	?	?	?

3.2 using S/Mime

o	1) OL2010 W7_64	2) OL2013 W8_64	3) OL2016 W7_32	4) OL2016 W8_64	5) OL2016 W10_64
a) Verify signature	?	?	?	?	?
b) Decrypt	?	?	?	?	?
c) Verify + Decrypt	?	?	?	?	?

4. Sender is Thunderbird, receiver is Outlook

4.1 using OpenPGP/Mime

o	1) OL2010 W7_64	2) OL2013 W8_64	3) OL2016 W7_32	4) OL2016 W8_64	5) OL2016 W10_64
a) Verify signature	?	?	?	?	?
b) Decrypt	?	?	?	?	?
c) Verify + Decrypt	?	?	?	?	?

4.2 using using S/MIME

o	1) OL2010 W7_64	2) OL2013 W8_64	3) OL2016 W7_32	4) OL2016 W8_64	5) OL2016 W10_64
a) Verify signature	?	?	?	?	?
b) Decrypt	?	?	?	?	?
c) Verify + Decrypt	?	?	?	?	?

-  ⇤ ← Revision 2 as of 2015-11-03 14:41:15 → 
  Size: 2274
  Editor: SeanEngelhardt
  Comment:
+   ← Revision 3 as of 2015-11-03 16:06:16 → ⇥
  Size: 9962
  Editor: SeanEngelhardt
  Comment: made clear-text test-cases
-Deletions are marked like this.
+Additions are marked like this.
 Line 1:
-## page was renamed from GpgOL/Development/Testcase
-Line 5:
+Line 4:
+<<TableOfContents(2)>>
-Line 34:
+Line 35:
-....
+The test cases described in plain text


1. Send a Mail from a Outlook Client to an Outlook Client.
**Use OpenPGP/Mime** for the signing process
# for Microsoft Outlook 2010 (On Windows 7 64bit)
** a) Sign the E-Mail
** b) Encrypt the E-Mail
** c) Sign and Encrypt the E-Mail in one step
** d) Verify signature of the E-Mail you just sent
** e) Decrypt the E-Mail you just sent
** f) Verify and decypt the email you just sent in one step
# for Microsoft Outlook 2013 (On Windows 8 64bit)
** a) Sign the E-Mail
** b) Encrypt the E-Mail
** c) Sign and Encrypt the E-Mail in one step
** d) Verify signature of the E-Mail you just sent
** e) Decrypt the E-Mail you just sent
** f) Verify and decypt the email you just sent in one step
# for Microsoft Outlook 2016 (On Windows 7 32bit)
** a) Sign the E-Mail
** b) Encrypt the E-Mail
** c) Sign and Encrypt the E-Mail in one step
** d) Verify signature of the E-Mail you just sent
** e) Decrypt the E-Mail you just sent
** f) Verify and decypt the email you just sent in one step
# for Microsoft Outlook 2016 (On Windows 8 64bit)
** a) Sign the E-Mail
** b) Encrypt the E-Mail
** c) Sign and Encrypt the E-Mail in one step
** d) Verify signature of the E-Mail you just sent
** e) Decrypt the E-Mail you just sent
** f) Verify and decypt the email you just sent in one step
# for Microsoft Outlook 2016 (On Windows 10 64bit)
** a) Sign the E-Mail
** b) Encrypt the E-Mail
** c) Sign and Encrypt the E-Mail in one step
** d) Verify signature of the E-Mail you just sent
** e) Decrypt the E-Mail you just sent
** f) Verify and decypt the email you just sent in one step


2. Send a Mail from a Outlook Client to an Outlook Client.
**Use S/Mime** for the signing process
# for Microsoft Outlook 2010 (On Windows 7 64bit)
** a) Sign the E-Mail
** b) Encrypt the E-Mail
** c) Sign and Encrypt the E-Mail in one step
** d) Verify signature of the E-Mail you just sent
** e) Decrypt the E-Mail you just sent
** f) Verify and decypt the email you just sent in one step
# for Microsoft Outlook 2013 (On Windows 8 64bit)
** a) Sign the E-Mail
** b) Encrypt the E-Mail
** c) Sign and Encrypt the E-Mail in one step
** d) Verify signature of the E-Mail you just sent
** e) Decrypt the E-Mail you just sent
** f) Verify and decypt the email you just sent in one step
# for Microsoft Outlook 2016 (On Windows 7 32bit)
** a) Sign the E-Mail
** b) Encrypt the E-Mail
** c) Sign and Encrypt the E-Mail in one step
** d) Verify signature of the E-Mail you just sent
** e) Decrypt the E-Mail you just sent
** f) Verify and decypt the email you just sent in one step
# for Microsoft Outlook 2016 (On Windows 8 64bit)
** a) Sign the E-Mail
** b) Encrypt the E-Mail
** c) Sign and Encrypt the E-Mail in one step
** d) Verify signature of the E-Mail you just sent
** e) Decrypt the E-Mail you just sent
** f) Verify and decypt the email you just sent in one step
# for Microsoft Outlook 2016 (On Windows 10 64bit)
** a) Sign the E-Mail
** b) Encrypt the E-Mail
** c) Sign and Encrypt the E-Mail in one step
** d) Verify signature of the E-Mail you just sent
** e) Decrypt the E-Mail you just sent
** f) Verify and decypt the email you just sent in one step


3. Send an E-Mail with KMail to an Outlook Client. Check if the Outlook Client can perform the following tasks without errors
# Perform the Steps using OpenPGP/Mime
## Microsoft Outlook 2010 (On Windows 7 64bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step
## Microsoft Outlook 2013 (On Windows 8 64bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step
## Microsoft Outlook 2016 (On Windows 7 32bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step
## Microsoft Outlook 2016 (On Windows 8 64bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step
## Microsoft Outlook 2016 (On Windows 10 64bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step
# Perform the Steps using S/Mime
## Microsoft Outlook 2010 (On Windows 7 64bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step
## Microsoft Outlook 2013 (On Windows 8 64bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step
## Microsoft Outlook 2016 (On Windows 7 32bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step
## Microsoft Outlook 2016 (On Windows 8 64bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step
## Microsoft Outlook 2016 (On Windows 10 64bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step


4. Send an E-Mail with Thunderbird to an Outlook Client. Check if the Outlook Client can perform the following tasks without errors
# Perform the Steps using OpenPGP/Mime
## Microsoft Outlook 2010 (On Windows 7 64bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step
## Microsoft Outlook 2013 (On Windows 8 64bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step
## Microsoft Outlook 2016 (On Windows 7 32bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step
## Microsoft Outlook 2016 (On Windows 8 64bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step
## Microsoft Outlook 2016 (On Windows 10 64bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step
# Perform the Steps using S/Mime
## Microsoft Outlook 2010 (On Windows 7 64bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step
## Microsoft Outlook 2013 (On Windows 8 64bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step
## Microsoft Outlook 2016 (On Windows 7 32bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step
## Microsoft Outlook 2016 (On Windows 8 64bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step
## Microsoft Outlook 2016 (On Windows 10 64bit)
*** a) Verify the signature
*** b) Decrypt the mail
*** c) Verify and Decrypt the E-Mail in one step
-Line 39:
+Line 207:
-|send|=OL2010 W7_64 |=OL2013 W8_64 |=OL2016 W7_32 | =OL2016 W8_64 | =OL2016 W10_64
|=Signing| ? | ? | ? | ? | ? |
|=Encrypting| ? | ? | ? | ? | ? |
|=Sig+Enc| ? | ? | ? | ? | ? |


=== consigner is the same client

|receive|=OL2010 W7_64 |=OL2013 W8_64 |=OL2016 W7_32 | =OL2016 W8_64 | =OL2016 W10_64
|=Verify signature| ? | ? | ? | ? | ? |
|=Decrypt| ? | ? | ? | ? | ? |
|=Verify + Decrypt| ? | ? | ? | ? | ? |

=== consigner is KMail

|receive|=OL2010 W7_64 |=OL2013 W8_64 |=OL2016 W7_32 | =OL2016 W8_64 | =OL2016 W10_64
|=Verify signature| ? | ? | ? | ? | ? |
|=Decrypt| ? | ? | ? | ? | ? |
|=Verify + Decrypt| ? | ? | ? | ? | ? |

=== consigner is thunderbird

|receive|=OL2010 W7_64 |=OL2013 W8_64 |=OL2016 W7_32 | =OL2016 W8_64 | =OL2016 W10_64
|=Verify signature| ? | ? | ? | ? | ? |
|=Decrypt| ? | ? | ? | ? | ? |
|=Verify + Decrypt| ? | ? | ? | ? | ? |
+=== 1. Sender and receiver is Outlook on Windows using OpenPGP/Mime

|o|=1) OL2010 W7_64 |=2) OL2013 W8_64 |=3) OL2016 W7_32 | =4) OL2016 W8_64 | =5) OL2016 W10_64
|=a) Signing| ? | ? | ? | ? | ? |
|=b) Encrypting| ? | ? | ? | ? | ? |
|=c) Sig+Enc| ? | ? | ? | ? | ? |
|=d) Verify signature| ? | ? | ? | ? | ? |
|=e) Decrypt| ? | ? | ? | ? | ? |
|=f) Verify + Decrypt| ? | ? | ? | ? | ? |


=== 2. Sender and receiver is Outlook on Windows using S/Mime

|o|=1) OL2010 W7_64 |=2) OL2013 W8_64 |=3) OL2016 W7_32 | =4) OL2016 W8_64 | =5) OL2016 W10_64
|=a) Signing| ? | ? | ? | ? | ? |
|=b) Encrypting| ? | ? | ? | ? | ? |
|=c) Sig+Enc| ? | ? | ? | ? | ? |
|=d) Verify signature| ? | ? | ? | ? | ? |
|=e) Decrypt| ? | ? | ? | ? | ? |
|=f) Verify + Decrypt| ? | ? | ? | ? | ? |


=== 3. Sender is KMail receiver is Outlook

==== 3.1 using OpenPGP/Mime

|o|=1) OL2010 W7_64 |=2) OL2013 W8_64 |=3) OL2016 W7_32 | =4) OL2016 W8_64 | =5) OL2016 W10_64
|=a) Verify signature| ? | ? | ? | ? | ? |
|=b) Decrypt| ? | ? | ? | ? | ? |
|=c) Verify + Decrypt| ? | ? | ? | ? | ? |

==== 3.2 using S/Mime

|o|=1) OL2010 W7_64 |=2) OL2013 W8_64 |=3) OL2016 W7_32 | =4) OL2016 W8_64 | =5) OL2016 W10_64
|=a) Verify signature| ? | ? | ? | ? | ? |
|=b) Decrypt| ? | ? | ? | ? | ? |
|=c) Verify + Decrypt| ? | ? | ? | ? | ? |


=== 4. Sender is Thunderbird, receiver is Outlook

==== 4.1 using OpenPGP/Mime
|o|=1) OL2010 W7_64 |=2) OL2013 W8_64 |=3) OL2016 W7_32 | =4) OL2016 W8_64 | =5) OL2016 W10_64
|=a) Verify signature| ? | ? | ? | ? | ? |
|=b) Decrypt| ? | ? | ? | ? | ? |
|=c) Verify + Decrypt| ? | ? | ? | ? | ? |

==== 4.2 using using S/MIME
|o|=1) OL2010 W7_64 |=2) OL2013 W8_64 |=3) OL2016 W7_32 | =4) OL2016 W8_64 | =5) OL2016 W10_64
|=a) Verify signature| ? | ? | ? | ? | ? |
|=b) Decrypt| ? | ? | ? | ? | ? |
|=c) Verify + Decrypt| ? | ? | ? | ? | ? |