Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Support for Case Deletion #33831

Open
wants to merge 48 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
48 commits
Select commit Hold shift + click to select a range
3570ec2
Add support for deleting individual cases (dry run only)
minhaminha Oct 25, 2023
580b5a1
Modify UI based on feedback (list structure, extra deletion confirmat…
minhaminha Nov 3, 2023
556d299
More UI changes based on feedback (list reorganization, using list-gr…
minhaminha Nov 17, 2023
5f1c4c4
Reconstruct form name if xmlns_to_name returns the xmlns
minhaminha Nov 21, 2023
707da7d
Add support for deleting cases and forms (no more dry run) + add tests
minhaminha Dec 1, 2023
8847642
Add line in task to delete old enough soft deleted cases + change wai…
minhaminha Dec 4, 2023
52b9443
Merge branch 'master' into ml/case-deletion
minhaminha Dec 4, 2023
870f8ca
add es_test decorator
minhaminha Dec 4, 2023
f78631c
Make small changes based on PR feedback
minhaminha Dec 5, 2023
d96c8a9
Refactor get_cases_and_forms_for_deletion in 3 separate functions + a…
minhaminha Dec 6, 2023
4e36269
Use attrs classes instead of complex dictionaries to pass around disp…
minhaminha Dec 8, 2023
3481e4f
Further break down get_case_and_display_data
minhaminha Dec 13, 2023
588b377
Disable hard deletion of eligible data
minhaminha Dec 13, 2023
e91c974
Undo delete task changes
minhaminha Dec 14, 2023
53852ac
Integrate case deletion into form deletion if the form had created an…
minhaminha Jan 25, 2024
f40be4b
Add more tests for form deletion, new case deletion util functions
minhaminha Jan 25, 2024
3c00173
Various refactors based on PR feedback
minhaminha Feb 8, 2024
fe480b3
Refactor tests + add more tests
minhaminha Feb 9, 2024
0f5e737
Refactor main case walk function into two smaller class methods, upda…
minhaminha Feb 13, 2024
2ee8572
Remove case hard deletion method (in favor of eventual tombstoning me…
minhaminha Feb 13, 2024
52c42bb
Merge branch 'master' into ml/case-deletion
minhaminha Feb 15, 2024
98dd353
small nit changes
minhaminha Feb 23, 2024
86f5deb
Make walk_through_case_forms always returns a dict
minhaminha Feb 23, 2024
18fd6ba
Add in memory caching for forms and case blocks
minhaminha Feb 26, 2024
9c65a10
Make sure forms are actually cached in TempFormCache, remove domain a…
minhaminha Mar 4, 2024
83872d4
Text changes + fix escaping issue
minhaminha Mar 5, 2024
76c6760
Merge branch 'master' into ml/case-deletion
minhaminha Mar 7, 2024
e5b25f8
Make small changes (remove manual escaping, add more tests about the …
minhaminha Mar 11, 2024
09ad4ff
Add TempCaseCache and tests, add cleanup to tests that create forms a…
minhaminha Mar 12, 2024
b93a8cb
Simplify test to save only once
minhaminha Mar 12, 2024
cf79b54
Merge branch 'master' into ml/case-deletion
minhaminha Mar 22, 2024
d3197eb
Fix bug caused by returned form list if there is no create form
minhaminha Mar 22, 2024
53f3877
Fix bug that raises a 404 when getting apps without an app_id
minhaminha Mar 25, 2024
1e49ef8
Minor text and code change suggestions
minhaminha Mar 25, 2024
e1fc391
Fix 404 raising bug when trying to fetch deleted forms + small refact…
minhaminha Mar 26, 2024
e5a2f9b
Fix bug that prevents bulk imports from able to delete
minhaminha Mar 26, 2024
9538140
Fix form order for large deletions
minhaminha Mar 28, 2024
55d63a3
Make form and case soft deletion all or nothing + prevent same forms …
minhaminha Mar 29, 2024
37d3f06
Minor fix
minhaminha Mar 29, 2024
23f27df
Small typo fix
minhaminha Mar 29, 2024
2565d07
Small typo fix part 2
minhaminha Apr 1, 2024
3a9d827
Reverse sorted list so latest form is archived first
minhaminha Apr 1, 2024
04bdb7b
Add test covering bulk form archive error handling
minhaminha Apr 2, 2024
1b1b86d
Typo fix in success message
minhaminha Apr 3, 2024
1b4b297
Make soft_delete_cases_and_forms a celery task (needs more work)
minhaminha Apr 3, 2024
c9b879e
Merge branch 'master' into ml/case-deletion
minhaminha Apr 3, 2024
b062be8
Merge branch 'master' into ml/case-deletion
gherceg May 28, 2024
bfa16b7
Merge branch 'master' into ml/case-deletion
minhaminha Jun 25, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
185 changes: 185 additions & 0 deletions corehq/apps/reports/standard/cases/case_data.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@
)
from django.utils.decorators import method_decorator
from django.utils.html import format_html
from django.utils.safestring import mark_safe
from django.utils.translation import get_language
from django.utils.translation import gettext as _
from django.utils.translation import gettext_lazy
Expand All @@ -24,6 +25,7 @@
from django_prbac.utils import has_privilege
from memoized import memoized

from casexml.apps.case import const
from casexml.apps.case.cleanup import close_case, rebuild_case_from_forms
from casexml.apps.case.mock import CaseBlock
from casexml.apps.case.templatetags.case_tags import case_inline_display
Expand Down Expand Up @@ -76,6 +78,7 @@
from corehq.apps.users.models import HqPermissions
from corehq.form_processor.exceptions import CaseNotFound
from corehq.form_processor.interfaces.dbaccessors import LedgerAccessors
from corehq.form_processor.interfaces.processor import FormProcessorInterface
from corehq.form_processor.models import (
CommCareCase,
UserRequestedRebuild,
Expand Down Expand Up @@ -555,6 +558,188 @@ def close_case_view(request, domain, case_id):
return HttpResponseRedirect(reverse('case_data', args=[domain, case_id]))


@location_safe
@require_case_view_permission
@require_permission(HqPermissions.edit_data)
def get_cases_and_forms_for_deletion(request, domain, case_id):
millerdev marked this conversation as resolved.
Show resolved Hide resolved
delete_cases = set() # actual cases to be soft deleted (not order dependent since it'll come after forms)
delete_forms = set() # actual forms to be soft deleted (order dependent - to be figured out)

# these are just for forming the disclaimer message
cases = {} # should be case_id: {form_id: the rest of the string} format is ' (action) > Case (action), etc'
case_names = {} # just case_id: case_name
reopened_cases = {}
affected_cases = {}

update_actions = [const.CASE_ACTION_INDEX,
const.CASE_ACTION_UPDATE,
const.CASE_ACTION_ATTACHMENT,
const.CASE_ACTION_COMMTRACK,
const.CASE_ACTION_REBUILD]

def get_forms_for_deletion_from_case(case, subcase_count):
redirect = False
delete_cases.add(case.case_id)
if len(delete_cases) > 10 or subcase_count + 1 > 3:
redirect = True
messages.error(request, _("Deleting this case would delete too many related cases. "
"Please delete some of this cases' subcases before attempting"
"to delete this case."))
return redirect
if case.case_id not in cases:
cases[case.case_id] = {}
case_names[case.case_id] = case.name
case_xforms = case.xform_ids

for form_id in case_xforms:
delete_forms.add(form_id)
form_object = XFormInstance.objects.get_form(form_id, domain)
# confirm this is doing what I intend for it to do
case_db = FormProcessorInterface(domain).casedb_cache(
domain=domain,
load_src="process_stock",
millerdev marked this conversation as resolved.
Show resolved Hide resolved
)
touched_cases = FormProcessorInterface(domain).get_cases_from_forms(case_db, [form_object])

case_actions = {}

for touched_id in touched_cases:
case_object = safely_get_case(request, domain, touched_id)
snopoke marked this conversation as resolved.
Show resolved Hide resolved
meta = touched_cases[touched_id]
if touched_id == case.case_id:
case_actions['current'] = list(meta.actions) # do I need to make it a list?
elif touched_id not in delete_cases:
if touched_id not in case_actions:
case_actions[case_object.name] = list(meta.actions)
if const.CASE_ACTION_CREATE in meta.actions and touched_id != case.case_id:
subcase_count += 1
get_forms_for_deletion_from_case(case_object, subcase_count)
subcase_count -= 1
millerdev marked this conversation as resolved.
Show resolved Hide resolved
if const.CASE_ACTION_CLOSE in meta.actions:
# theoretically only one form should ever close a case?
reopened_cases[touched_id] = 'closed by: ' + form_id
# this needs to be expanded on / different actions handled separately
if any(action in meta.actions for action in update_actions):
if touched_id not in affected_cases:
affected_cases[touched_id] = {}
affected_cases[touched_id][form_id] = ' (' + ', '.join(list(meta.actions)) + ')'
case_names[touched_id] = case_object.name

form_string = ' (' + ', '.join(case_actions.pop('current')) + ')'
if len(case_actions):
form_string += ' > '
case_strings = []
for case_name in case_actions:
case_strings.append('{} ({})'.format(case_name, ', '.join(case_actions[case_name])))
form_string += ', '.join(case_strings)
cases[case.case_id][form_id] = form_string
return redirect

case_instance = safely_get_case(request, domain, case_id)
subcase_count = 0
should_redirect = get_forms_for_deletion_from_case(case_instance, subcase_count)
if should_redirect:
return {}, True
millerdev marked this conversation as resolved.
Show resolved Hide resolved

def get_case_link(caseid):
url = reverse('case_data', args=[domain, caseid])
return '<a href="{}"> {} </a>'.format(url, case_names[caseid])
millerdev marked this conversation as resolved.
Show resolved Hide resolved

def get_form_link(formid, form_string):
url = reverse('render_form_data', args=[domain, formid])
return '<a href="{}"> {} </a>'.format(url, formid) + form_string
millerdev marked this conversation as resolved.
Show resolved Hide resolved

delete_case_message_block = []
for case in cases:
delete_case_message_block.append(
get_case_link(case) + " " + "<ul>{}</ul>".format(
"".join([f"<li>{get_form_link(form, cases[case][form])}</li>" for form in cases[case]])
)
)
reopened_case_message_block = []
for case in reopened_cases:
reopened_case_message_block.append(
get_case_link(case) + " " + reopened_cases[case]
)
affected_case_message_block = []
for case in affected_cases:
if case in delete_cases:
continue
affected_case_message_block.append(
get_case_link(case) + " " + "<ul>{}</ul>".format(
"".join([f"<li>{get_form_link(form, affected_cases[case][form])}</li>"
for form in affected_cases[case]])
)
)

delete_list_string = _("The following Cases and their forms will be deleted: <ul>{}</ul>")\
.format("".join(delete_case_message_block))
if reopened_case_message_block:
reopened_list_string = _("The following Cases will be reopened: <ul>{}</ul>")\
.format("".join(reopened_case_message_block))
delete_list_string += reopened_list_string
if affected_case_message_block:
affected_list_string = _("The following Cases will be affected, but not deleted: <ul>{}</ul>") \
.format("".join(affected_case_message_block))
delete_list_string += affected_list_string

return {
'formatted_delete_list': mark_safe(delete_list_string),
'case_delete_list': delete_cases,
'form_delete_list': delete_forms,
}, False


@location_safe
class DeleteCaseView(BaseProjectReportSectionView):
urlname = 'soft_delete_case_view'
page_title = gettext_lazy('Delete Case and Related Forms')
template_name = 'reports/reportdata/case_delete.html'
delete_dict = {}

def dispatch(self, request, *args, **kwargs):
self.delete_dict, redirect = get_cases_and_forms_for_deletion(request, self.domain, self.case_id)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How long does this take to run? Wondering it would make sense to offload it to celery

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From my testing, It doesn’t take long at all. There’s also a limit on how many cases it can look at/return so the lag shouldn’t really be noticeable.

if redirect:
return HttpResponseRedirect(reverse('case_data', args=[self.domain, self.case_id]))
return super(DeleteCaseView, self).dispatch(request, *args, **kwargs)

@property
def case_id(self):
return self.kwargs['case_id']

@property
def domain(self):
return self.kwargs['domain']

@property
def page_url(self):
return reverse(self.urlname, args=(self.domain, self.case_id))

@property
def page_context(self):
context = {
"case_id": self.case_id,
}
context.update(self.delete_dict)
return context

def post(self, request, *args, **kwargs):
soft_delete_case(request, self.domain, None)
msg = "[DRY RUN] Nothing has been deleted, but it will eventually!"
messages.success(request, msg, extra_tags='html')
return HttpResponseRedirect(reverse('case_data', args=[self.domain, self.case_id]))


@require_permission(HqPermissions.edit_data)
@location_safe
def soft_delete_case(request, domain, delete_dict=None):
print("in soft_delete_case")
# to be expanded on later
# perform soft delete
# return success or error message
return delete_dict


@location_safe
@require_case_view_permission
@require_permission(HqPermissions.edit_data)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -285,6 +285,12 @@ <h1>{% blocktrans %}Section: {{ section_id }}{% endblocktrans %}</h1>
</button>
</form>
{% endif %}
{% if not is_usercase %}
<a class="btn btn-danger pull-left" href="{% url 'soft_delete_case_view' domain case_id %}">
<i class="fa fa-archive"></i>
{% trans 'Delete Case and Related Forms' %}
</a>
{% endif %}
</div>
</div>
{% endif %}
Expand Down
40 changes: 40 additions & 0 deletions corehq/apps/reports/templates/reports/reportdata/case_delete.html
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
{% extends "hqwebapp/bootstrap3/base_section.html" %}
{% load case_tags %}
{% load hq_shared_tags %}
{% load i18n %}
{% load proptable_tags %}
{% load timezone_tags %}

{% block head %} {{ block.super }}
<link rel="stylesheet" type="text/css" href="{% static "hqwebapp/css/proptable.css" %}">
{% endblock %}

{% requirejs_main 'reports/js/case_details' %}

{% block page_content %}
<div>
<h2>
Case Deletion
</h2>
<h4>
Please review the listed items before proceeding with the case deletion.
</h4>
</br>
<p>
{{ formatted_delete_list }}
</p>
</br>
<div class="clearfix form-actions">
<div class="col-sm-12">
<a class="btn btn-default pull-left" href="{% url 'case_data' domain case_id %}">
{% trans 'Cancel' %}
</a>
<form class="form form-horizontal disable-on-submit" method="post" id="delete-cases-forms">
{% csrf_token %}
<button type="submit" class="btn btn-danger">{% trans "Permanently Delete Cases and Forms" %}</button>
</form>
</div>
</div>
</div>

{% endblock %}
3 changes: 3 additions & 0 deletions corehq/apps/reports/urls.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@
case_property_names,
case_xml,
close_case_view,
DeleteCaseView,
download_case_history,
edit_case_view,
export_case_transactions,
Expand Down Expand Up @@ -109,6 +110,8 @@
url(r'^case_data/(?P<case_id>[\w\-]+)/close/$', close_case_view, name="close_case"),
url(r'^case_data/(?P<case_id>[\w\-]+)/undo-close/(?P<xform_id>[\w\-:]+)/$',
undo_close_case_view, name="undo_close_case"),
url(r'^case_data/(?P<case_id>[\w\-]+)/delete_case_view/$',
DeleteCaseView.as_view(), name=DeleteCaseView.urlname),
url(r'^case_data/(?P<case_id>[\w\-]+)/export_transactions/$',
export_case_transactions, name="export_case_transactions"),
url(r'^case_data/(?P<case_id>[\w\-]+)/(?P<xform_id>[\w\-:]+)/$', case_form_data, name="case_form_data"),
Expand Down