-
Notifications
You must be signed in to change notification settings - Fork 0
/
IndividualTicketTesting.html
169 lines (133 loc) · 9.33 KB
/
IndividualTicketTesting.html
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
<!DOCTYPE html>
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>Individual Ticket Testing</title>
<link rel="stylesheet" href="_static/bootstrap-sphinx.css" type="text/css" />
<link rel="stylesheet" href="_static/pygments.css" type="text/css" />
<link rel="stylesheet" href="_static/custom.css" type="text/css" />
<script type="text/javascript" id="documentation_options" data-url_root="./" src="_static/documentation_options.js"></script>
<script type="text/javascript" src="_static/jquery.js"></script>
<script type="text/javascript" src="_static/underscore.js"></script>
<script type="text/javascript" src="_static/doctools.js"></script>
<script type="text/javascript" src="_static/js/jquery-1.11.0.min.js"></script>
<script type="text/javascript" src="_static/js/jquery-fix.js"></script>
<script type="text/javascript" src="_static/bootstrap-3.3.7/js/bootstrap.min.js"></script>
<script type="text/javascript" src="_static/bootstrap-sphinx.js"></script>
<link rel="index" title="Index" href="genindex.html" />
<link rel="search" title="Search" href="search.html" />
<link rel="next" title="Writing Performance Tests" href="WritingPerformanceTests.html" />
<link rel="prev" title="Unit Test Good Practice" href="UnitTestGoodPractice.html" />
<script>
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-59110517-1', 'auto');
ga('send', 'pageview');
</script>
</head><body>
<div id="navbar" class="navbar navbar-default ">
<div class="container">
<div class="navbar-header">
<!-- .btn-navbar is used as the toggle for collapsed navbar content -->
<button type="button" class="navbar-toggle" data-toggle="collapse" data-target=".nav-collapse">
<span class="icon-bar"></span>
<span class="icon-bar"></span>
<span class="icon-bar"></span>
</button>
<a class="navbar-brand" href="http://www.mantidproject.org"><img src="_static/Mantid_Logo_Transparent.png">
</a>
<span class="navbar-text navbar-version pull-left"><b>master</b></span>
</div>
<div class="collapse navbar-collapse nav-collapse">
<ul class="nav navbar-nav">
<li class="divider-vertical"></li>
<li><a href="index.html">Home</a></li>
<li><a href="http://download.mantidproject.org">Download</a></li>
<li><a href="http://www.mantidproject.org">Wiki</a></li>
<li><a href="http://docs.mantidproject.org">User Documentation</a></li>
<li><a href="http://www.mantidproject.org/Contact">Contact Us</a></li>
</ul>
<form class="navbar-form navbar-right" action="search.html" method="get">
<div class="form-group">
<input type="text" name="q" class="form-control" placeholder="Search" />
</div>
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
</div>
<div class="container">
<div class="row">
<div class="col-md-12 content">
<div class="section" id="individual-ticket-testing">
<span id="individualtickettesting"></span><h1>Individual Ticket Testing<a class="headerlink" href="#individual-ticket-testing" title="Permalink to this headline">¶</a></h1>
<p>An important step in our <a class="reference internal" href="GitWorkflow.html#gitworkflow"><span class="std std-ref">development workflow</span></a> is the testing of individual issues/tickets after the development on them is complete, and before the code is merges into the master branch. Developers pick one from <a class="reference external" href="https://github.com/mantidproject/mantid/pulls">the list</a> of completed issues and perform a number of verification steps on it. The mechanics of testing a pull request (e.g. the git commands to use) are described <a class="reference internal" href="GitWorkflow.html#gitworkflow"><span class="std std-ref">here</span></a>. This page is concerned with the aspects that should be considered in deciding whether a pull request should be recommended to merge or sent back to the developer for further work. <em>There should be very little reluctance to reopen a ticket even for minor issues.</em></p>
<div class="section" id="code-review">
<h2>Code Review<a class="headerlink" href="#code-review" title="Permalink to this headline">¶</a></h2>
<p>The code changes should be manually reviewed (the github compare view is ideal for this). A couple of pieces on the value of code review can be found at <a class="reference external" href="http://scientopia.org/blogs/goodmath/2011/07/06/things-everyone-should-do-code-review">scientopia</a> and <a class="reference external" href="http://www.codinghorror.com/blog/2006/01/code-reviews-just-do-it.html">codinghorror</a>.</p>
<ul class="simple">
<li>The primary aim is to find bugs that the developer and tests so far have not spotted.</li>
<li>But also consider whether the code is ‘clean’, well-structured and easy to read/maintain.</li>
<li>Part of this is that:<ul>
<li>There should be are no compiler (or doxygen) warnings coming from any modified classes</li>
<li>The code conforms to our <a class="reference internal" href="Standards/MantidStandards.html#mantidstandards"><span class="std std-ref">coding standards</span></a>.</li>
</ul>
</li>
<li>Unit tests (or system tests if more appropriate) should be checked that they:<ul>
<li>Exist and give adequate coverage (see <a class="reference internal" href="UnitTestGoodPractice.html#unittestgoodpractice"><span class="std std-ref">unit testing practices</span></a>).</li>
<li>If the ticket is fixing a bug there should be a test that makes sure we don’t have to fix the same bug again!</li>
<li>Do not load real data (data loading algorithms get a free pass on this one).</li>
<li>Leave the system in the same state that they found it (i.e. clean up).</li>
<li>Have a performance test, if appropriate.</li>
</ul>
</li>
<li>Check that any user documentation is adequate and that there are release notes. In the case of new algorithms, there should be an accompanying <code class="docutils literal notranslate"><span class="pre">*.rst</span></code> file that has been added, containing an explanation of what exactly the algorithm does along with Python usage examples.</li>
</ul>
</div>
<div class="section" id="functional-testing">
<h2>Functional Testing<a class="headerlink" href="#functional-testing" title="Permalink to this headline">¶</a></h2>
<p>The first thing to note is that this should <strong>not</strong> just be a quick check of whatever the ticket says it does. Testing should be as much, if not more, about making sure the code <em>does not do what it’s not supposed to do</em> as that it <em>does do what it’s supposed to</em>.</p>
<ul class="simple">
<li>All of the builds pass</li>
<li>The developer should have included instructions in the ticket of how to test things work.</li>
<li>But, as noted above, don’t just do that – also try to break it: click random buttons on GUIs, give unexpected/invalid inputs, etc.</li>
<li>Note down what you did in the ticket, and the platform you did it on.</li>
</ul>
<p>If all the requirements have been met and documented approve the PR using <a class="reference external" href="https://help.github.com/articles/about-pull-request-reviews/">GitHub’s review mechanism</a>.</p>
</div>
<div class="section" id="gatekeeper">
<h2>Gatekeeper<a class="headerlink" href="#gatekeeper" title="Permalink to this headline">¶</a></h2>
<p>The <code class="docutils literal notranslate"><span class="pre">@mantidproject/gatekeepers</span></code> group is who is meant to merge pull requests into master. This is done by social contract. A gatekeeper can <code class="docutils literal notranslate"><span class="pre">merge</span></code> code if:</p>
<ul class="simple">
<li>Green tick on the last build indicating all automated testing has succeeded</li>
<li>Adequate tests, both success and failure cases have been performed</li>
<li>There is comment on the code being reviewed</li>
</ul>
</div>
</div>
</div>
</div>
</div>
<footer class="footer">
<div class="container">
<ul class="nav navbar-nav" style=" float: right;">
<li>
<a href="UnitTestGoodPractice.html" title="Previous Chapter: Unit Test Good Practice"><span class="glyphicon glyphicon-chevron-left visible-sm"></span><span class="hidden-sm hidden-tablet">« Unit Test Goo...</span>
</a>
</li>
<li>
<a href="WritingPerformanceTests.html" title="Next Chapter: Writing Performance Tests"><span class="glyphicon glyphicon-chevron-right visible-sm"></span><span class="hidden-sm hidden-tablet">Writing Perfo... »</span>
</a>
</li>
<li><a href="#">Back to top</a></li>
</ul>
<p>
</p>
</div>
</footer>
</body>
</html>