|
Size: 6796
Comment: Move gpgol test report to new page
|
Size: 8266
Comment: GpgOL test plan improvements
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 68: | Line 68: |
| ==== 2.2 Move Mails | ==== 2.2 Move mails |
| Line 85: | Line 85: |
| Disable the GpgOL plugin in Outlook (via COM add-ons). Select a signed/encrypted mail. Check that the mail cannot decrypted/verify. |
Disable the GpgOL plugin in Outlook (via COM add-ons). Select a OpenPGP signed/encrypted mail. Check that the mail cannot decrypt/verify. |
| Line 89: | Line 88: |
TODO === Testsuite 3. Interoperability tests (Kontact Mail) Send crypto mails from Kontact Mail (KMail) to Outlook (GpgOL). ==== 3.1 Using PGP/MIME |
Open a a S/~M~I~M~E sign/encrypted mail (which was signed/encrypted with GpgOL). Deactivate S/MIME in GpgOL's option dialog. Close and reopen the mail. Check that Outlook's S/MIME implementation can decrypt/verify this mail. ==== 2.9 Close Outlook Open a signed/encrypted mail. Close Outlook. Answer "Yes" if Outlook wants to save the changes. Restart Outlook. Check with an other mail client (via IMAP) //before// and //after// restart of Outlook that the mail is never unencrypted on server. ==== 2.10 Unknown recipient Try to encrypt a message to a unknown email address. Check that Kleopatra cannot match a suitable certificate. ==== 2.11 Cannot decrypt Send an encrypted email to yourself but select an other certificate for encryption (not the own certificate). Check that GpgOL cannot decrypt the received message. ==== 2.12 Cannot sign Remove your certificate (incl. secret key). Try to sign a new message. Check that GpgOL cannot sign the message. ==== 2.13 Cannot encrypt Remove your certificate (if still available). Try to encrypt a new message to yourself. Check that GpgOL cannot encrypt the message. ==== 2.14 Signature with wrong address Create or import new secret key which does not match to Outlooks email address. Send a signed mail to yourself but select the new certificate for signing. Verify the signature. Check the warning that the email address not contained in the certificate. === Testsuite 3: Interoperability tests (Outlook's built-in S/MIME) ==== 3.1 Receive S/MIME mails with GpgOL Send S/~M~I~M~E crypto mails from Outlook (with it's built-in S/MIME implementation) to an other Outlook (with GpgOL). a) sign and verify\\ b) encrypt and decrypt\\ c) sign/encrypt (with attachment*) and decrypt/verify (and check attachment) ==== 3.1 Send S/MIME mails with GpgOL Send S/~M~I~M~E crypto mails from Outlook (with GpgOL) to an other Outlook (with it's built-in S/MIME implementation). a) sign and verify\\ b) encrypt and decrypt\\ c) sign/encrypt (with attachment*) and decrypt/verify (and check attachment) === Testsuite 4: Interoperability tests (Kontact Mail) Send crypto mails between Kontact Mail (KMail) and Outlook (with GpgOL). ==== 4.1 PGP/MIME |
| Line 104: | Line 142: |
| ==== 3.2 Using S/MIME | ==== 4.2 S/MIME |
| Line 112: | Line 150: |
| ==== 3.3 Using no-mime-PGP | ==== 4.3 INLINE/PGP |
| Line 118: | Line 156: |
| === Testsuite 4. Interoperability tests (Thunderbird) Send crypto mails from Thunderbird (Enigmail) to Outlook (GpgOL). ==== 4.1 Using PGP/MIME |
=== Testsuite 5: Interoperability tests (Thunderbird) Send crypto mails between Thunderbird (with Enigmail) and Outlook (with GpgOL). ==== 5.1 PGP/MIME |
| Line 130: | Line 168: |
| ==== 4.2 Using S/MIME (with Thunderbird's S/MIME implementation) | ==== 5.2 S/MIME (with Thunderbird's S/MIME implementation) |
| Line 138: | Line 176: |
| ==== 4.3 Using no-mime-PGP | ==== 5.3 INLINE/PGP |
| Line 145: | Line 183: |
| == Test matrix Legend:\\ ? = untested\\ x = passed\\ F = failed ** +++ Tested ~GpgOL version: X.X.X +++** |Outlook |=2010 (32) |= 2013 (64) |=2016 (32) |= 2016 (64) |= 2016 (64) |= .. |Windows |=7 (64) |= 8 (64) |=7 (32) |= 8 (64) |= 10 (64) |= .. |Testsuite 1: OL/OL | |1.1.a | ? | ? | ? | ? | ? | .. | |1.1.b | ? | ? | ? | ? | ? | .. | |1.1.c | ? | ? | ? | ? | ? | .. | |1.2.a | ? | ? | ? | ? | ? | .. | |1.2.b | ? | ? | ? | ? | ? | .. | |1.2.c | ? | ? | ? | ? | ? | .. | |Testsuite 2: KMail/OL | |2.1.a | ? | ? | ? | ? | ? | .. | |2.1.b | ? | ? | ? | ? | ? | .. | |2.1.c | ? | ? | ? | ? | ? | .. | |2.1.d | ? | ? | ? | ? | ? | .. | |2.1.e | ? | ? | ? | ? | ? | .. | |2.1.f | ? | ? | ? | ? | ? | .. | |2.2.a | ? | ? | ? | ? | ? | .. | |2.2.b | ? | ? | ? | ? | ? | .. | |2.2.c | ? | ? | ? | ? | ? | .. | |2.2.d | ? | ? | ? | ? | ? | .. | |2.2.e | ? | ? | ? | ? | ? | .. | |2.2.f | ? | ? | ? | ? | ? | .. | |2.3.a | ? | ? | ? | ? | ? | .. | |2.3.b | ? | ? | ? | ? | ? | .. | |2.3.c | ? | ? | ? | ? | ? | .. | |Testsuite 3: Thunderbird/OL | |3.1.a | ? | ? | ? | ? | ? | .. | |3.1.b | ? | ? | ? | ? | ? | .. | |3.1.c | ? | ? | ? | ? | ? | .. | |3.1.d | ? | ? | ? | ? | ? | .. | |3.1.e | ? | ? | ? | ? | ? | .. | |3.1.f | ? | ? | ? | ? | ? | .. | |3.2.a | ? | ? | ? | ? | ? | .. | |3.2.b | ? | ? | ? | ? | ? | .. | |3.2.c | ? | ? | ? | ? | ? | .. | |3.2.d | ? | ? | ? | ? | ? | .. | |3.2.e | ? | ? | ? | ? | ? | .. | |3.2.f | ? | ? | ? | ? | ? | .. | |3.3.a | ? | ? | ? | ? | ? | .. | |3.3.b | ? | ? | ? | ? | ? | .. | |3.3.c | ? | ? | ? | ? | ? | .. | |
=== Testsuite 6: Interoperability tests (GPGTools) Send crypto mails between Apple Mail (with GPG for Mail plugin from GPGTools) and Outlook (with GpgOL). ==== 6.1 PGP/MIME a) sign (Apple Mail) -> verify (GpgOL)\\ b) encrypt (Apple Mail) -> decrypt (GpgOL)\\ c) sign/encrypt with attachment (Apple Mail) -> decrypt/verify and check attachment (GpgOL)\\ d) answer to (a): sign (GpgOL) -> verify (Apple Mail)\\ e) answer to (b): encrypt (GpgOL) -> decrypt (Apple Mail)\\ f) answer to (c): sign/encrypt with attachment (GpgOL) -> decrypt/verify and check attachment (Apple Mail) ==== 6.2 S/MIME a) sign (Apple Mail) -> verify (GpgOL)\\ b) encrypt (Apple Mail) -> decrypt (GpgOL)\\ c) sign/encrypt with attachment (Apple Mail) -> decrypt/verify and check attachment (GpgOL)\\ d) answer to (a): sign (GpgOL) -> verify (Apple Mail)\\ e) answer to (b): encrypt (GpgOL) -> decrypt (Apple Mail)\\ f) answer to (c): sign/encrypt with attachment (GpgOL) -> decrypt/verify and check attachment (Apple Mail) |
GpgOL Test Plan
Status: in progress
Introduction
The purpose of this test plan is to ensure GpgOL works well on all supported Microsoft Windows operating systems with all supported versions of Microsoft Outlook.
Test environment
Choose one combination of the following target platforms supported by GpgOL.
Microsoft Outlook:
- 2003 (32bit)
- 2007 (32bit)
- 2010 (32 or 64bit)
- 2013 (32 or 64bit)
- 2016 (32 or 64bit)
Operation system:
- Windows Vista (32 or 64bit)
- Windows 7 (32 or 64bit)
- Windows 8 (32 or 64bit)
- Windows 10 (32 or 64bit)
For interoperability tests additional mail clients will be tested together with Outlook/GpgOL:
- Thunderbird with Enigmail-Add-On (on Windows, MacOS or GNU/Linux)
- Apple Mail with GPGTools (on MacOS)
- Kontact Mail (KMail) enterprise 35 (on GNU/Linux)
- Kontact Mail (KMail) of KDE 4.14 (on GNU/Linux)
It is recommended to use a clean virtual test environment, e.g. with VirtualBox.
Preconditions
- email account A on IMAP/SMTP server
- email account B on Exchange Server 2013 or later
- OpenPGP public key (*.asc or *.gpg) and private key (*.asc or *.gpg)
- S/MIME certificate chain (*.pem) and private key (*.p12)
Tests
Testsuite 1: GpgOL basic crypto tests
Send crypto mails from/to same Outlook account.
1.1 PGP/MIME
Send mails with PGP/MIME:
a) sign and verify
b) encrypt and decrypt
c) sign/encrypt (with attachment*) and decrypt/verify (and check attachment)
1.2 S/MIME
Send mails with S/MIME:
a) sign and verify
b) encrypt and decrypt
c) sign/encrypt (with attachment*) and decrypt/verify (and check attachment)
(*) Attachment should contain umlauts in filename (e.g. fooäüö.txt)
Testsuite 2: GpgOL extended tests
2.1 Exchange Server
Run all tests from testsuite 1 via Exchange Server.
2.2 Move mails
Move some encrypted mails from inbox into an other folder. Check with an other IMAP client that all mails are always encrypted.
2.3 Drafts
Create new message. Mark message with "sign" and "encrypt". Close and save mail as draft. Open draft again. Check if the message is still marked with "sign" and "encrypt".
2.4 Cancel crypto operation
Create a mew message. Mark as sign/encrypt. Click send. Cancel certificate selection dialog. Check if the message is not sent.
2.5 Sent messages
Send a signed/encrypted message. Check the sent folder ("Sent messages") if the mail is still signed/encrypted.
2.6 Reply
Reply to an encrypted mail. Check if the quoted text contains the decrypted message.
2.7 Disable GpgOL
Disable the GpgOL plugin in Outlook (via COM add-ons). Select a OpenPGP signed/encrypted mail. Check that the mail cannot decrypt/verify.
2.8 Deactivate GpgOL's S/MIME (GpgSM)
Open a a S/MIME sign/encrypted mail (which was signed/encrypted with GpgOL). Deactivate S/MIME in GpgOL's option dialog. Close and reopen the mail. Check that Outlook's S/MIME implementation can decrypt/verify this mail.
2.9 Close Outlook
Open a signed/encrypted mail. Close Outlook. Answer "Yes" if Outlook wants to save the changes. Restart Outlook.
Check with an other mail client (via IMAP) before and after restart of Outlook that the mail is never unencrypted on server.
2.10 Unknown recipient
Try to encrypt a message to a unknown email address. Check that Kleopatra cannot match a suitable certificate.
2.11 Cannot decrypt
Send an encrypted email to yourself but select an other certificate for encryption (not the own certificate). Check that GpgOL cannot decrypt the received message.
2.12 Cannot sign
Remove your certificate (incl. secret key). Try to sign a new message. Check that GpgOL cannot sign the message.
2.13 Cannot encrypt
Remove your certificate (if still available). Try to encrypt a new message to yourself. Check that GpgOL cannot encrypt the message.
2.14 Signature with wrong address
Create or import new secret key which does not match to Outlooks email address. Send a signed mail to yourself but select the new certificate for signing. Verify the signature. Check the warning that the email address not contained in the certificate.
Testsuite 3: Interoperability tests (Outlook's built-in S/MIME)
3.1 Receive S/MIME mails with GpgOL
Send S/MIME crypto mails from Outlook (with it's built-in S/MIME implementation) to an other Outlook (with GpgOL).
a) sign and verify
b) encrypt and decrypt
c) sign/encrypt (with attachment*) and decrypt/verify (and check attachment)
3.1 Send S/MIME mails with GpgOL
Send S/MIME crypto mails from Outlook (with GpgOL) to an other Outlook (with it's built-in S/MIME implementation).
a) sign and verify
b) encrypt and decrypt
c) sign/encrypt (with attachment*) and decrypt/verify (and check attachment)
Testsuite 4: Interoperability tests (Kontact Mail)
Send crypto mails between Kontact Mail (KMail) and Outlook (with GpgOL).
4.1 PGP/MIME
a) sign (KMail) -> verify (GpgOL)
b) encrypt (KMail) -> decrypt (GpgOL)
c) sign/encrypt with attachment (KMail) -> decrypt/verify and check attachment (GpgOL)
d) answer to (a): sign (GpgOL) -> verify (KMail)
e) answer to (b): encrypt (GpgOL) -> decrypt (KMail)
f) answer to (c): sign/encrypt with attachment (GpgOL) -> decrypt/verify and check attachment (KMail)
4.2 S/MIME
a) sign (KMail) -> verify (GpgOL)
b) encrypt (KMail) -> decrypt (GpgOL)
c) sign/encrypt with attachment (KMail) -> decrypt/verify and check attachment (GpgOL)
d) answer to (a): sign (GpgOL) -> verify (KMail)
e) answer to (b): encrypt (GpgOL) -> decrypt (KMail)
f) answer to (c): sign/encrypt with attachment (GpgOL) -> decrypt/verify and check attachment (KMail)
4.3 INLINE/PGP
a) sign (KMail) -> verify (GpgOL)
b) encrypt (KMail) -> decrypt (GpgOL)
c) sign/encrypt without attachment (KMail) -> decrypt/verify (GpgOL)
Testsuite 5: Interoperability tests (Thunderbird)
Send crypto mails between Thunderbird (with Enigmail) and Outlook (with GpgOL).
5.1 PGP/MIME
a) sign (TB) -> verify (GpgOL)
b) encrypt (TB) -> decrypt (GpgOL)
c) sign/encrypt with attachment (TB) -> decrypt/verify and check attachment (GpgOL)
d) answer to (a): sign (GpgOL) -> verify (TB)
e) answer to (b): encrypt (GpgOL) -> decrypt (TB)
f) answer to (c): sign/encrypt with attachment (GpgOL) -> decrypt/verify and check attachment (TB)
5.2 S/MIME (with Thunderbird's S/MIME implementation)
a) sign (TB) -> verify (GpgOL)
b) encrypt (TB) -> decrypt (GpgOL)
c) sign/encrypt with attachment (TB) -> decrypt/verify and check attachment (GpgOL)
d) answer to (a): sign (GpgOL) -> verify (TB)
e) answer to (b): encrypt (GpgOL) -> decrypt (TB)
f) answer to (c): sign/encrypt with attachment (GpgOL) -> decrypt/verify and check attachment (TB)
5.3 INLINE/PGP
a) sign (TB) -> verify (GpgOL)
b) encrypt (TB) -> decrypt (GpgOL)
c) sign/encrypt with attachment (TB) -> decrypt/verify and check attachment (GpgOL)
Testsuite 6: Interoperability tests (GPGTools)
Send crypto mails between Apple Mail (with GPG for Mail plugin from GPGTools) and Outlook (with GpgOL).
6.1 PGP/MIME
a) sign (Apple Mail) -> verify (GpgOL)
b) encrypt (Apple Mail) -> decrypt (GpgOL)
c) sign/encrypt with attachment (Apple Mail) -> decrypt/verify and check attachment (GpgOL)
d) answer to (a): sign (GpgOL) -> verify (Apple Mail)
e) answer to (b): encrypt (GpgOL) -> decrypt (Apple Mail)
f) answer to (c): sign/encrypt with attachment (GpgOL) -> decrypt/verify and check attachment (Apple Mail)
6.2 S/MIME
a) sign (Apple Mail) -> verify (GpgOL)
b) encrypt (Apple Mail) -> decrypt (GpgOL)
c) sign/encrypt with attachment (Apple Mail) -> decrypt/verify and check attachment (GpgOL)
d) answer to (a): sign (GpgOL) -> verify (Apple Mail)
e) answer to (b): encrypt (GpgOL) -> decrypt (Apple Mail)
f) answer to (c): sign/encrypt with attachment (GpgOL) -> decrypt/verify and check attachment (Apple Mail)