Allows working copy of the Portal

[wesleybl]

This is part of: plone/volto#5284

Allows working copy services to be accessed in the Portal. Also returns working copy data in the Portal serialization.

In Plone 6.0, this PR should be tested together with the following PRs:

plone/plone.app.iterate#128
plone/Products.CMFEditions#113

In Plone 6.1, the plone.app.iterate PR should be replaced with:

plone/plone.app.iterate#130

---

📚 Documentation preview 📚: https://plonerestapi--1823.org.readthedocs.build/

[mister-roboto]

@wesleybl thanks for creating this Pull Request and helping to improve Plone!

TL;DR: Finish pushing changes, pass all other checks, then paste a comment:

@jenkins-plone-org please run jobs

To ensure that these changes do not break other parts of Plone, the Plone test suite matrix needs to pass, but it takes 30-60 min. Other CI checks are usually much faster and the Plone Jenkins resources are limited, so when done pushing changes and all other checks pass either start all Jenkins PR jobs yourself, or simply add the comment above in this PR to start all the jobs automatically.

Happy hacking!

[wesleybl]

I tried to do a checkout test of the Portal in this class:

plone.restapi/src/plone/restapi/tests/test_services_workingcopy.py

Line 12 in f209cf5

class TestWorkingCopyEndpoint(unittest.TestCase):

The test is as follows:

    def test_workingcopy_checkout_checkin_portal(self):
        # We create the working copy
        response = self.api_session.post(
            "/@workingcopy",
        )

        self.assertEqual(response.status_code, 201)

But I am getting error 500. I went to debug and the error is not in the restapi code. But in Zope. The error occurs when trying to execute this commit:

https://github.com/zopefoundation/Zope/blob/5.10/src/ZPublisher/WSGIPublisher.py#L187

The error is a POSKeyError. I executed the commit outside the try and the traceback was:

*** transaction.interfaces.TransactionFailedError: An operation previously failed, with traceback:

  File "/usr/lib/python3.12/threading.py", line 1030, in _bootstrap
    self._bootstrap_inner()
  File "/usr/lib/python3.12/threading.py", line 1073, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.12/threading.py", line 1010, in run
    self._target(*self._args, **self._kwargs)
  File "/home/user/.buildout/eggs/cp312/waitress-2.1.2-py3.12.egg/waitress/task.py", line 84, in handler_thread
    task.service()
  File "/home/user/.buildout/eggs/cp312/waitress-2.1.2-py3.12.egg/waitress/channel.py", line 428, in service
    task.service()
  File "/home/user/.buildout/eggs/cp312/waitress-2.1.2-py3.12.egg/waitress/task.py", line 168, in service
    self.execute()
  File "/home/user/.buildout/eggs/cp312/waitress-2.1.2-py3.12.egg/waitress/task.py", line 434, in execute
    app_iter = self.channel.server.application(environ, start_response)
  File "/home/user/.buildout/eggs/cp312/WebTest-3.0.0-py3.12.egg/webtest/http.py", line 82, in wrapper
    return self.test_app(environ, start_response)
  File "/home/user/.buildout/eggs/cp312/Zope-5.10-py3.12.egg/ZPublisher/httpexceptions.py", line 30, in __call__
    return self.application(environ, start_response)
  File "/home/user/.buildout/eggs/cp312/Zope-5.10-py3.12.egg/ZPublisher/WSGIPublisher.py", line 390, in publish_module
    with transaction_pubevents(request, response):
  File "/usr/lib/python3.12/contextlib.py", line 144, in __exit__
    next(self.gen)
  File "/home/user/.buildout/eggs/cp312/Zope-5.10-py3.12.egg/ZPublisher/WSGIPublisher.py", line 187, in transaction_pubevents
    tm.commit()
  File "/home/user/.buildout/eggs/cp312/transaction-4.0-py3.12.egg/transaction/_manager.py", line 254, in commit
    return self.manager.commit()
  File "/home/user/.buildout/eggs/cp312/transaction-4.0-py3.12.egg/transaction/_manager.py", line 133, in commit
    return self.get().commit()
  File "/home/user/.buildout/eggs/cp312/transaction-4.0-py3.12.egg/transaction/_transaction.py", line 280, in commit
    t, v, tb = self._saveAndGetCommitishError()
  File "/home/user/.buildout/eggs/cp312/transaction-4.0-py3.12.egg/transaction/_transaction.py", line 273, in commit
    self._commitResources()
  File "/home/user/.buildout/eggs/cp312/transaction-4.0-py3.12.egg/transaction/_transaction.py", line 456, in _commitResources
    raise v.with_traceback(tb)
  File "/home/user/.buildout/eggs/cp312/transaction-4.0-py3.12.egg/transaction/_transaction.py", line 430, in _commitResources
    rm.commit(self)
  File "/home/user/.buildout/eggs/cp312/ZODB-5.8.1-py3.12.egg/ZODB/Connection.py", line 492, in commit
    self._commit_savepoint(transaction)
  File "/home/user/.buildout/eggs/cp312/ZODB-5.8.1-py3.12.egg/ZODB/Connection.py", line 1033, in _commit_savepoint
    obj._p_estimated_size = len(data)
    ^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/git/buildout.coredev/src/Products.CMFPlone/Products/CMFPlone/Portal.py", line 68, in __setattr__
    if self._tree is not None and name in self:
       ^^^^^^^^^^
  File "/home/user/.buildout/eggs/cp312/ZODB-5.8.1-py3.12.egg/ZODB/Connection.py", line 787, in setstate
    p, serial = self._storage.load(oid)
                ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.buildout/eggs/cp312/ZODB-5.8.1-py3.12.egg/ZODB/mvccadapter.py", line 164, in load
    r = self._storage.loadBefore(oid, self._start)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.buildout/eggs/cp312/ZODB-5.8.1-py3.12.egg/ZODB/DemoStorage.py", line 229, in loadBefore
    return self.base.loadBefore(oid, tid)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.buildout/eggs/cp312/ZODB-5.8.1-py3.12.egg/ZODB/DemoStorage.py", line 229, in loadBefore
    return self.base.loadBefore(oid, tid)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/.buildout/eggs/cp312/ZODB-5.8.1-py3.12.egg/ZODB/DemoStorage.py", line 229, in loadBefore
    return self.base.loadBefore(oid, tid)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  [Previous line repeated 2 more times]
  File "/home/user/.buildout/eggs/cp312/ZODB-5.8.1-py3.12.egg/ZODB/utils.py", line 281, in __call__
    return func(*args, **kw)
           ^^^^^^^^^^^^^^^^^
  File "/home/user/.buildout/eggs/cp312/ZODB-5.8.1-py3.12.egg/ZODB/MappingStorage.py", line 171, in loadBefore
    raise ZODB.POSException.POSKeyError(oid)
ZODB.POSException.POSKeyError: 0x2a33e463be38bb08

When Plone is running normally this error does not occur, only in testing. Could there be some missing test configuration?

@mauritsvanrees @davisagli @d-maurer @dataflake

[d-maurer]

Wesley Barroso Lopes wrote at 2024-10-8 06:14 -0700:

... File "/home/user/.buildout/eggs/cp312/ZODB-5.8.1-py3.12.egg/ZODB/MappingStorage.py", line 171, in loadBefore raise ZODB.POSException.POSKeyError(oid) ZODB.POSException.POSKeyError: 0x2a33e463be38bb08

You try to load an object state which is not there. Your use of a `DemoStorage` (i.e. a RAM based storage) suggests a test setup with few objects. The oid `0x2a33e463be38bb08` is huge for such a setup. I cannot tell you more. I likely would try to find out where this oid comes from.

[wesleybl]

The oid 0x2a33e463be38bb08 is huge for such a setup.

@d-maurer the functionality I'm implementing clones the portal with pickle. See:

https://github.com/plone/Products.CMFEditions/blob/c80bc31af46bff45fee2908878b1f01190fda8d8/Products/CMFEditions/ArchivistTool.py#L210-L229

Usually the Portal is a large object. Could it be that the test configuration is not supporting the clone size?

In real life, would I have to optimize the portal clone, to clone only the attributes that are really needed?

[d-maurer]

Wesley Barroso Lopes wrote at 2024-10-8 08:12 -0700:

> The oid `0x2a33e463be38bb08` is huge for such a setup. @d-maurer the functionality I'm implementing clones the portal with pickle. See: https://github.com/plone/Products.CMFEditions/blob/c80bc31af46bff45fee2908878b1f01190fda8d8/Products/CMFEditions/ArchivistTool.py#L210-L229 Usually the Portal is a large object.

Your `oid` is huge; it indicates a setup much larger than a typical Plone site.

Could it be that the test configuration is not supporting the clone size?

I would not expect a `POSKeyError` indicating a RAM (storage) shortage.

In real life, would I have to optimize the portal clone, to clone only the attributes that are really needed?

I am not sure that cloning a portal will work out of the box. At least, the cloned catalog will contain entries refering to the source (and not only entries refering to the clone). They might be other problems. Keep fingers crossed.

[wesleybl]

@d-maurer there is already a treatment to prevent child objects of the Portal from being placed in the clone:

https://github.com/plone/Products.CMFEditions/blob/13fe34186f7294113f5c14bac6e8f78170868569/Products/CMFEditions/StandardModifiers.py#L322-L347

The child objects are replaced by an object of the VersionAwareReference class. So my theory that the clone was getting big because of the child objects is wrong.

Checking the traceback, I noticed that the error occurs in the Portal class, in this line:

https://github.com/plone/Products.CMFPlone/blob/6.0.13/Products/CMFPlone/Portal.py#L68

So I printed self before that line. The result was:

...
<PloneSite at plone>
<PloneSite at plone>
<Products.CMFPlone.Portal.PloneSite object at 0x7f697d4581a0 oid 0x1d86b25a16bc20b5 in <ZODB.Connection.Connection object at 0x7f697d4ddb80>>

In other words, the object that caused the error was this:

<Products.CMFPlone.Portal.PloneSite object at 0x7f697d4581a0 oid 0x1d86b25a16bc20b5 in <ZODB.Connection.Connection object at 0x7f697d4ddb80>>

When we had the object:

<PloneSite at plone>

the error did not occur. Does this give any clue as to what might be causing the error?

[d-maurer]

Wesley Barroso Lopes wrote at 2024-10-8 12:16 -0700:

... <Products.CMFPlone.Portal.PloneSite object at 0x7f697d4581a0 oid 0x1d86b25a16bc20b5 in <ZODB.Connection.Connection object at 0x7f697d4ddb80>>

OIDs (= "Object IDs") are assigned sequentially; this means, an OID roughly gives a hint on how many objects are in the ZODB (at least if objects have not been massively deleted). The OID "0x1d86b25a16bc20b5" corresponds to "2.127.583.973.977.366.709" or roughly 2 * 10^18. I cannot believe that you really have a ZODB with so many objects. I fear something is very wrong with your setup (and the OIDs are wrong which would correspond to `POSKeyError`s).

... When we had the object: <PloneSite at plone>

This is the `repr` format for an acquisition wrapper; this means, here you do not yet access the base persistent object but the acquisition wrapper wrapping the object at location `plone`. The `<Products.CMFPlone.Portal.PloneSite object at 0x7f697d4581a0 oid 0x1d86b25a16bc20b5 in <ZODB.Connection.Connection object at 0x7f697d4ddb80>` follows the `repr` format for a persistent object (when derived classes do not define `repr` differently). This means that you access here the base portal object (no longer acquisition wrapped). If `pobj` is a persistent object (or an acquisition wrapper thereof), you get its OID with `pobj._p_oid` and its ZODB connection with `poid._p_jar`. You can check whether a ZODB connection `conn` knows about an OID `oid` with `conn[oid]`. This will give you the (unwrapped) object with `oid` or a `POSKeyError` (if the object is not known).

[d-maurer]

Your use of a DemoStorage (i.e. a RAM based storage) suggests a test setup with few objects. The oid 0x2a33e463be38bb08 is huge for such a setup.

I have been partially wrong: DemoStorage is not a (pure) RAM based storage. Instead it allows to put a RAM based storage (more precisely a MappingStorage) on top of any storage (in particular a non RAM based storage).

Looking more closely at your traceback:
The recursive call DemoStorage.py", line 229, in loadBefore suggests that you have several DemoStorages layered on top of one another. The final traceback line from MappingStorage suggests that you have a MappingStorage at the base of your DemoStorage stack. This would mean that your storage is indeed fully RAM based.

Checking the code, I see that DemoStorage chooses its initial oid randomly: self._next_oid = random.randint(1, 1 << 62). This explains the huge oid value. Using randomness at this point may lead to non-deterministic failures, however, e.g. that the same oid is used for different objects. I suggest you replace line 135 of ZODB.DemoStorage (self._next_oid = random.randint(1, 1 << 62)) by self._next_oid = self.base._next_oid + 0x100000 and check whether this lets your test succeed.

[d-maurer]

I have created zopefoundation/ZODB#401

[wesleybl]

I suggest you replace line 135 of ZODB.DemoStorage (self._next_oid = random.randint(1, 1 << 62)) by self._next_oid = self.base._next_oid + 0x100000 and check whether this lets your test succeed.

After changing the line, I get the error:

AttributeError: 'MappingStorage' object has no attribute '_next_oid'. Did you mean: 'new_oid'?

I did a dir on self.base:

['__class__', '__delattr__', '__dict__', '__dir__', '__doc__', '__eq__', '__format__', '__ge__', '__getattribute__', '__getstate__', '__gt__', '__hash__', '__implemented__', '__init__', '__init_subclass__', '__le__', '__len__', '__lt__', '__module__', '__name__', '__ne__', '__new__', '__providedBy__', '__provides__', '__reduce__', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__', '__subclasshook__', '__weakref__', '_commit_lock', '_data', '_last_pack', '_lock', '_ltid', '_oid', '_opened', '_transaction', '_transactions', 'checkCurrentSerialInTransaction', 'cleanup', 'close', 'getName', 'getSize', 'getTid', 'history', 'isReadOnly', 'iterator', 'lastTransaction', 'load', 'loadBefore', 'loadSerial', 'new_oid', 'not_in_transaction', 'opened', 'pack', 'registerDB', 'sortKey', 'store', 'tpc_abort', 'tpc_begin', 'tpc_finish', 'tpc_transaction', 'tpc_vote']

[d-maurer]

Wesley Barroso Lopes wrote at 2024-10-9 05:57 -0700:

> I suggest you replace line 135 of ZODB.DemoStorage (self._next_oid = random.randint(1, 1 << 62)) by self._next_oid = self.base._next_oid + 0x100000 and check whether this lets your test succeed. After changing the line, I get the error: ``` AttributeError: 'MappingStorage' object has no attribute '_next_oid'. Did you mean: 'new_oid'? ```

My fault! I had not checked the base implementation. Looking at `DemoStorage.new_oid`: the `_next_oid` is just used as a potential OID candidate; `new_oid` checks that it is currently not used; if it is, then a new candidate is randomly chosen. Thus, at least if the base storage is not independently changed (unlikely in your situation), `DemoStorage`'s OID generation should be safe. Almost surely, my assumption was wrong that the huge OID is related to the problem you have observed.

[wesleybl]

@d-maurer I debugged a bit and found out that this error occurs when we have a savepoint and a Plone Site object is involved. It occurs when the code enters here:

https://github.com/zopefoundation/ZODB/blob/5.8.1/src/ZODB/Connection.py#L486-L494

However, the error only occurs in the test and when there is a portal object involved. Even in the test, this code works, if there is no modified Plone Site object.

In the normal execution of an instance, the code does not enter the if, because self._savepoint_storage is None. self._savepoint_storage is not filled, because we have no resources here:

https://github.com/zopefoundation/transaction/blob/4.0/src/transaction/_transaction.py#L618

So the savepoint is not actually created. But in the test we have resources and the savepoint code is executed.

In fact, if I return after this line, the test works.

Do we have a limitation of DemoStorage running savepoint code when we have a Plone Site involved?

[d-maurer]

File "/home/user/.buildout/eggs/cp312/ZODB-5.8.1-py3.12.egg/ZODB/Connection.py", line 1033, in _commit_savepoint
    obj._p_estimated_size = len(data)
    ^^^^^^^^^^^^^^^^^^^^^
  File "/home/user/git/buildout.coredev/src/Products.CMFPlone/Products/CMFPlone/Portal.py", line 68, in __setattr__
    if self._tree is not None and name in self:
       ^^^^^^^^^^

New hypothesis: The code cited above tries to update the object's size estimation (obj._p_estimated_size = ...). This is an assignment to one of the special persistency attributes (identified by an _p_ name prefix); it should happen without access to any other attributes of obj. But, apparently obj._tree is accessed and results in a ZODB load which apparently fails.

I suggest you try the following:
in Products.CMFPlone.Portal.__setattr__ you replace

        if self._tree is not None and name in self:

by

      if not name.startswith("_p_") and self._tree is not None and name in self:

Apparently the special __setattr__ is there to allow (content) items to be stored in a tree structure (providing support for a huge number of child items). Because the id of content items must not start with _, the startswith("_p_") can be replaced by startswith("_").

[wesleybl]

I suggest you try the following:
in Products.CMFPlone.Portal.__setattr__ you replace

   if self._tree is not None and name in self:

by

 if not name.startswith("_p_") and self._tree is not None and name in self:

@d-maurer This worked! Thanks!

I'll try to create a PR to fix this.

[davisagli]

@wesleybl Could you add a test?

[d-maurer]

David Glick wrote at 2024-10-30 13:45 -0700:

@davisagli commented on this pull request. @wesleybl Could you add a test?

It is very difficult to create a reliable test for the original szenario (portal copy). One could create a test following the decription in "zopefoundation/ZODB#402". It would use a mockup class with a `__setattr__` read accessing a (normal) instance attribute. The test would instantiate a class instance, create a savepoint, use the instance as attribute of an object stored inside the ZODB and then `_p_deactivate` the instance. A following `commit` should show the problem (because the instance state can only be loaded from the temporary savepoint storage no longer available during the savepoint commit). If `__setattr__` is fixed to not access a normal attribute for an assignment to `_p_estimated_size`, the error should go away. The test would not directly verify the fixed `PortalSite.__setattr__`. Instead, it would indirectly show that the approach used there is effective for this kind of `savepoint` commit problem.

[davisagli]

I meant a functional test that the new REST API endpoints in this PR are working as expected.

[wesleybl]

@davisagli for the test to work, we first need a Plone release with: plone/Products.CMFPlone#4026

I need to backport this to Plone 6.0.

[d-maurer]

***@***.*** wrote at 2024-10-30 22:06 +0100:

David Glick wrote at 2024-10-30 13:45 -0700: ***@***.*** Could you add a test? It is very difficult to create a reliable test for the original szenario (portal copy).

Maybe, I was too pessimistic.

One could create a test following the decription in "zopefoundation/ZODB#402". It would use a mockup class with a `__setattr__` read accessing a (normal) instance attribute. The test would instantiate a class instance, create a savepoint, use the instance as attribute of an object stored inside the ZODB and then `_p_deactivate` the instance. A following `commit` should show the problem (because the instance state can only be loaded from the temporary savepoint storage no longer available during the savepoint commit). If `__setattr__` is fixed to not access a normal attribute for an assignment to `_p_estimated_size`, the error should go away.

We likely could use `PortalSite` as the mockup class `_tree` is the (normal) attribute accessed in `__setattr__`).

[d-maurer]

David Glick wrote at 2024-10-30 14:09 -0700:

I meant a functional test that the new REST API endpoints in this PR are working as expected.

The former observations (works in production, fails in test) show that special conditions are necessary for a failure. Therefore, a test may indicate success even though the problem is still there. Based on my current understanding, I suppose that during the test accidentally some objects are ghostified which are not in production. Therefore, I would say that a test should enforce that an essential object has been ghostified.

[wesleybl]

@davisagli Should the github actions tests pass even if we don't have the package versions with the other PRs? Should we expect a new Plone version with the other PRs?