Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Harris Matrix: test first example from URAP #2399

Closed
lbestock opened this issue Nov 24, 2023 · 66 comments
Closed

Harris Matrix: test first example from URAP #2399

lbestock opened this issue Nov 24, 2023 · 66 comments
Assignees
Labels
kiosk a kiosk issue (not a filemaker issue)

Comments

@lbestock
Copy link
Collaborator

Testing Unit FA much as #2398 tests an MKAP unit.
FA

Initial notes:

FA-001 and FA-002 are bonded. That is a "same time as" relationship and they are not on the same level. Furthermore FA-024 is known directly (now that I updated the relations) to be later than FA-001. Shouldn't FA-001 be lower than FA-024, then?

In general the levels are very poor indicators of chronology. Only the relations between individual nodes are useful on that front, the overall needs me to move things up or down all over the place. I note only the FA-001/2 thing because I thought this should have caught that with a relationship thus defined. (This is a problem with the HM generator, too. I

There is a clear test case of the A-B-C issue in FA-025, FA-026, FA-027. If I am reading this purely for chronology, which I am, it does not bother me that this is represented as simple and linear while the physical relations were not.

@urapadmin
Copy link
Collaborator

Here is the new FA as an SVG:
FA

@urapadmin urapadmin added the kiosk a kiosk issue (not a filemaker issue) label Nov 24, 2023
@urapadmin
Copy link
Collaborator

In general the levels are very poor indicators of chronology. Only the relations between individual nodes are useful on that front, the overall needs me to move things up or down all over the place.

Is that a critique of the concept of the Harris Matrix, then I agree, or do you think some of those loci could have been placed in a better layer by the algorithm on the basis of the available information?

@lbestock
Copy link
Collaborator Author

I need to test if it would be possible on the basis of available information. But it is clear to me that this could only be used in interpretation itself, not as an illustration of interpretation, if one cannot adjust the chronological levels/vertical positions of things manually. For instance I suspect strongly enough that it wants representation that FA-056 and FA-027 are contemporary. That is not algorithmically derivable. But I would like to move the whole of the left most column (Sondage 1) down so that FA-027 is at the same level as FA-056 (in Sondage 2).

Note to myself: in terms of deriving things algorithmically, your test should be: You know that FA-005 is later than FA-002. Is there any way Kiosk should have known that? That should be complicated enough...

@urapadmin
Copy link
Collaborator

well, I "strongly suspect" is problematic enough. My suggestion would be that one can add interpretational relationships that would be visualized differently from the others and I can even imagine that one could add different sets of them. But lets not make it too complicated. If I give you the ability to establish an interpretational relationship in the graph that re-renders the thing (only be aware that re-rendering can change the whole graph substantially) then you could add the algorithmic data that makes the nodes move to the layers you suspect they belong.

@lbestock
Copy link
Collaborator Author

Well but that is just it, right? Is the HM a representation of the known certain chronological implications of stratigraphic relations? Or is it a representation of the interpretation of chronological events? I want to be able to play around with an interpretational relationship and see what it does, but is that really complicated on your end?

I am currently struggling with an interesting case. FA-006 is a wall. All the walls were built before the floor was put down. The floor was put down in two phases, FA-023 is a mud layer, and then FA-024 is the bricks set in it. In the case of FA-006 we did not have a cut perpendicular to the wall, so we saw this only from the top, we could only record that FA-024 abuts the wall FA-006. I know from the relationship of those two floor phases to one another and how they must have been constructed, and can see in relation to other walls, that there is no way FA-023 can be the same date as FA-006 - the wall had to be there before any of the floor was built. I can only observe it for the top layer in this case. So I have an HM with FA-006 contemporary to FA-023, and that's not wrong algorithmically - they are both earlier than FA-024. But it is wrong and I know it to be so.

The relationship of the FA-024 floor to the walls is the place where I can best test the A-B-C issue. On a purely chronological level it's right everywhere else. But those are the major architectural phases of this unit and they touch each other and I really have to remind myself no, this does a different thing than a section drawing does.

@lbestock
Copy link
Collaborator Author

lbestock commented Nov 24, 2023

So now I have two interpretational relations I want to input if given that opportunity:
FA-027 is the same time as FA-056.
FA-006 is earlier than FA-023.

(I am editing this comment to reflect all the interpretational relations I want to put in, so I don't have to sort through the whole ticket later:

FA-027 is the same time as FA-056.
FA-006 is earlier than FA-023.
FA-006 is earlier than FA-004.
FA-039 is later than FA-002.
FA-039 is earlier than FA-005.
FA-050 is later than FA-002.
FA-050 is earlier than FA-005.
(Those last four should have a major difference on the HM because they reflect the fact that I interpret the walls of the room as having been built at different times and with leveling fill layers between - the walls were terracing. I want to see this with and without those FA-050 interpretational ones in there! Not clear to me if I need the last two algorithmically - might be enough to just get the FA-039 ones in there.)

To play with (I thought it was possible but was not sure, and it would rearrange a few things on the HM): FA-042 is equivalent to FA-042.

Oh also to play with: FA-027 and FA-046 were thought at one point to potentially be the same layer. I need to try that - to my eye looking at the current HM that is stratigraphically impossible but I am not confident in that at all and want to know what happens if it is the case.

@lbestock
Copy link
Collaborator Author

I would also add interpretationally that:
FA-006 is earlier than FA-004.

That is a less secure interpretation, however, so wow danger.

I still really like this wonderful puzzle of a unit. It was so much fun.

@urapadmin
Copy link
Collaborator

Well but that is just it, right? Is the HM a representation of the known certain chronological implications of stratigraphic relations? Or is it a representation of the interpretation of chronological events?

Or are there archaeologists with "formal training" in HM who do have expectations? Well, I don't know. But apart from that we can decide ourselves what we want it to be and how we want it to work. To me it seems much more useful as a tool to study your stratigraphy than as a somewhat static picture that tells you the stratigraphy (because it can't in part because there are many loci that cannot be put on one or the other layer). So playing around with it (coloring it, marking it, enriching it with other data from the record) would be interesting.

There is just one major caveat from my point of view: if I give you the opportunity to play around with your stratigraphy you might be tempted to make it fit and add interpretational relationships until it is more plausible than it actually should be.

But one can battle that part at least by visually distinguishing between relationships that have been recorded in the field or found otherwise physically compelling and suspected (or wished for :o)) relations added later.

@urapadmin
Copy link
Collaborator

urapadmin commented Nov 24, 2023

perhaps somebody wants to read the manual? One can get it for free at http://harrismatrix.com/about-the-book/ in many languages 🌵

@luizaogs has added thoughts of the same nature in #2398. Let's discuss the "interpretational relations" here in FA for the time being.

@luizaogs
Copy link
Collaborator

I like the idea of interpretational relations :)

@urapadmin
Copy link
Collaborator

While you guys are analyzing this please try to distill a few good archaeological cases out of this that show what I think is the main weakness of the harris matrix: Seemingly false precision. While I have to place a node (locus) on one of the layers often enough the data does not require it to be on exactly one and not any other layer. So the node seems to be contemporaneous with some other node but the data is not that clear. If was for a while thinking that one should draw the corridor of layers in which a locus exists relative to other layers. Could be a shaded background or so. But I think the Harris method does not do such a thing.

So if you run into actual archaeology that reveals the problem that I only see theoretically please write it down so that it can be explained in the class room.

@luizaogs
Copy link
Collaborator

Yeah I noted a few instances where I thought this had happened in the AA matrix in #2398 but I'll note more if I see them.

@urapadmin
Copy link
Collaborator

💤 Time for me. But this is fun.

@lbestock
Copy link
Collaborator Author

Musing by the fire.
Yes, the false precision is an issue and a way to capture the imprecision would be good.
However. I was initially annoyed that things that are unknown are landing at the latest of the places they could be, since I often knew or suspected they were earlier, and I am not annoyed by that anymore. On the contrary, if we can't find a way to indicate imprecision, then things definitely belong on the latest possible position the algorithm returns. Why? Because the implications of dating something too late are usually much smaller than the implications of dating something too early. When someone makes a mistake too early they write books rewriting history and huge energy is needed to correct the mistake. Dating something too late usually just means you ignore it for longer than warranted. I am reminded of how I teach students to clean loci and how uncomfortable they are scraping the top of something away before beginning to collect from it, and how the division between those who get and value the problem of contamination and those who don't divides those who have a prayer of doing this from those who don't. Also my FA-006 example reminds me that "archaeology is destruction" is not a bad thing. If we had taken out FA-023 and FA-024 up to the wall that would no longer be ambiguous in the records because we could have documented the relationship instead of inferred it.

I want interpretational relations for the HM. But where do they exist and how and how aside from the HM would they be seen and recorded? I'm very wary of having them in the recording in a way equivalent to the observed relations; the slippage where something recorded becomes something extant is scary to me. We have always been careful to separate the observation and interpretation for a reason. @urapadmin do you mean to give us these in a way that changes the matrix only, or that changes the recording?

@lbestock
Copy link
Collaborator Author

Another missing relation in the data that will make a difference:
FA-009 is above and thus later than FA-024.

(Take a look at FA-010 and FA-011 - if they are not contemporary with FA-009 something is weird, though there should not be any algorithmic way of determining that other than that they are all directly on top of FA-024.)

@lbestock
Copy link
Collaborator Author

Yeah wow.
FA-017 is later than FA-008 and FA-009 (abutting); and FA-024 (above).
Shoddy recording.

@lbestock
Copy link
Collaborator Author

(I want a quality control rule that nothing except bedrock or topsoil can possibly have only one locus relation, and even that would be suspect.)

I fixed those last ones but you might want to wait until I have gone through everything to run the HM again.

@lbestock
Copy link
Collaborator Author

Last not totally coherent thought before I go to bed:
I have been thinking of ambiguity in the places where there is a long vertical line. Those are obvious - the node can slide along that line. More insidious is that there is ambiguity in places that are denser, as in above, where FA-006 and FA-023 are on the same horizontal line and earlier than FA-024. Those are all short lines, they don't jump out at you as a place of ambiguity, but we know there should be three chronological steps there not two, so there is a whole horizontal section missing (not algorithmically, but archaeologically). So it's not just where things are phasing-wise, it's even how many phases there are, that get confused in that way. In a more minor way the case of equivalence between FA-027 and FA-05 is similar. There there would not be an increase in overall number of phases, but there needs to be a longer vertical line somewhere (where?) in the relations between layers in that sondage to get those into the same phase.

@lbestock
Copy link
Collaborator Author

Already with FA and certainly with anything that has more loci: an absolutely ideal HM UI would include the ability to search for and highlight a locus. I am doing so much going back and forth (I have the recording system open on two devices and sheets of paper flying all over the place too), so it's not just going from the matrix to the recording. I need to go the other direction, too. And then it takes me ages to find my locus.

@luizaogs
Copy link
Collaborator

luizaogs commented Nov 25, 2023

Already with FA and certainly with anything that has more loci: an absolutely ideal HM UI would include the ability to search for and highlight a locus. I am doing so much going back and forth (I have the recording system open on two devices and sheets of paper flying all over the place too), so it's not just going from the matrix to the recording. I need to go the other direction, too. And then it takes me ages to find my locus.

Oh yes, I second this if that’s possible to do. Even with AA yesterday which is much less complicated I kept losing loci and getting a headache going back and forth.

I mean, actually, an absolutely ideal HM UI would not only allow me to search for loci but also to go to those loci by clicking on them. It sounds like this will be in Kiosk rather than the recording app, yes? Is there a way to connect it to Q&V to pull up locus information that way?

@lbestock
Copy link
Collaborator Author

A case of what I might call compound false precision that is interesting:
Several small but disconnected walls on the west side of FA are almost certainly all of the same chronological phase - they defined the bins in which grain was stored. I think I can see in looking at the matrix that there is no actual reason for them not to be on the same level, and indeed three of the five of them are. But the two that are not on that level are very not on that level, and not on the same level as one another. The matrix has not misinterpreted any of the entered archaeological relations incorrectly but it has made choices about where to put things that pretty seriously distort the sense because they seem precise and different.
The loci in question are:
FA-008
FA-009
FA-010
FA-011
FA-034

I can add these to the "interpretational" list above, but I note them separately here because I think this is what you are asking for in terms of false precision?

A much more straightforward case is FA-058. Again no way for the algorithm to know. That is on the same level as FA-019, just for instance, and FA-058 is ancient and FA-019 is a modern looter's pit, so definitely not the same phase. But there is really no way an algorithm could know this. FA--58 was ancient but it is not a well-networked locus, being a floor on top of a floor and already exposed to the sky when we found it.

@urapadmin
Copy link
Collaborator

Okay, while we get many ideas on how to implement a Harris Matrix as a more interactive tool here, what I still need before I can do anything is:
Are the chronological relationships in FA (and AA) correct as far as the evidence allows an algorithm to infer chronology?

@urapadmin
Copy link
Collaborator

urapadmin commented Nov 25, 2023

A case of what I might call compound false precision that is interesting: Several small but disconnected walls on the west side of FA are almost certainly all of the same chronological phase ...

FA-008 FA-009 FA-010 FA-011 FA-034

I can add these to the "interpretational" list above, but I note them separately here because I think this is what you are asking for in terms of false precision?

A much more straightforward case is FA-058. Again no way for the algorithm to know. That is on the same level as FA-019, just for instance, and FA-058 is ancient and FA-019 is a modern looter's pit, so definitely not the same phase.

  • I will look into these examples and if it turns out to be random, I will see if there is any option for me to tweak the algorithm so that it consistently puts them up (which it should be doing). But I have very limited influence on the positioning of the nodes.

When you are finally done with FA and are absolutely convinced that the locus relations in the recording database are correct as far as possible (no more shoddy recording) I will go through the not insubstantial labour again and put them in the HM composer to get a final opinion on what that one spits out.

@lbestock
Copy link
Collaborator Author

lbestock commented Nov 25, 2023

On a sense level (I still have not caught this out as having any errors in applying the relations) FA-017 needs to be later than all those little walls just listed above.

@arch-kiosk arch-kiosk deleted a comment from lbestock Nov 25, 2023
@luizaogs
Copy link
Collaborator

Thinking of the trouble I had with the AA matrix, that seems very helpful.

@lbestock
Copy link
Collaborator Author

I do have a question about why in the cluster one FA-008 is treated as having a different relationship to FA-002. Or at least, it is the only one of the cluster whose abutting to FA-002 is not rendered, and I thought I must have missed the relation but at least in the current version (so many versions, one of the problems) it is already in there.

So. I like the grey arrows. Not too intrusive, provide the answer as to which things are actually known to be contemporary. I dislike that it crosses FA-023 up there - @urapadmin you just added these, or were they taken into account when rendering? I am ready to compare this with the proprietary HM generator, so you can go ahead and feed that beast.

@urapadmin
Copy link
Collaborator

urapadmin commented Nov 27, 2023

Oh look, I missed yet another relation: FA-008 also abuts FA-002. The clustering goes some way towards making it look more like the actual stratigraphy.

Could that be the reason why?
At least I do not have a contemporary relation between FA-008 and FA-002, which would change the graph quite a bit.

@lbestock
Copy link
Collaborator Author

lbestock commented Nov 27, 2023 via email

@urapadmin
Copy link
Collaborator

you might. I didn't. Yesterday when I made this I was still using the data from the day before. You must tell me when you actually synchronize a new relation.
However, I have done that now and very little changes as FA-008 -> FA-002 is transitive (and hence removed) because of FA-008 -> FA-004 -> FA-002.

@urapadmin
Copy link
Collaborator

urapadmin commented Nov 27, 2023

Here is the HM Composer version of that unit. To stay under the 48 node limit I had to cut out a few more boring relations:
5->22 -> 25 -> 27 -> 26 -> 28
39 -> 50 -> 52 -> 56 -> 57
31 -> 32 -> 33

I was trying to cut out edges that would have the least impact on the overall graph. The result is this:

image

As you can see, the Composer, too, eliminates the abutting of FA-008 and FA-002 on validation. So on that front we are consistent.

@urapadmin
Copy link
Collaborator

So: Supposing that I can get closer to the orthogonal lines of the hm composer (I will not avoid crossings as well as that one), how close are we here from the archaeological standpoint? Can you find advantages in a region?
Note how the HM Composer is treating the floating nodes (the ones that could be on any layer, like FA-036, FA-058, FA-030 and FA-029, but also FA-010 and FA-011).

@lbestock
Copy link
Collaborator Author

lbestock commented Dec 3, 2023

Comparing these is complicated; I'll make notes here as I go.

Are all the locus relations for FA-033 input for the HM? If so it has made a mistake; FA-033 is by direct relationship earlier than FA-030 and the HM composer has them on the same line.

Both Kiosk and HM put FA-019 and FA-003 on the top line. Those are ALL that are on the top line for HM, whereas Kiosk lifts a bunch of other things up. It's a mixed bag - FA-014 and FA-036 absolutely belong at the end chronologically and thus in the top layer. FA-058 does not. But neither of these could have known that. Both put FA-010 and FA-011 near the top. Those are amongst the loci we clustered above and brought into line with FA-008 and FA-009. Neither composer gets that without us clustering them.

One advantage of the orthogonal lines in HM that I hadn't appreciated is that it makes it easier for me to see which loci could slide up and down chronologically - made it easier to catch FA-033. But right now my feeling is that Kiosk actually does a better job getting loci with relationally indeterminate chronology into later positions, and while that is of course sometimes a mistake it is as said above a lesser problem than putting things too early.

That FA-024 is its own phase is the case in both.

@urapadmin
Copy link
Collaborator

Are all the locus relations for FA-033 input for the HM? If so it has made a mistake; FA-033 is by direct relationship earlier than FA-030 and the HM composer has them on the same line.

I missed that one. No bugs in the hm composer.

@urapadmin
Copy link
Collaborator

and as for nodes on the same layer: We must once and for all accept that unless there is a direct contemporary relation between two loci one must not read the fact that two loci share a layer as any indication that they are contemporary. That is really misreading a Harris Matrix. I wonder if Harris himself pointed that fact out in his book that everybody refuses to read.

@lbestock
Copy link
Collaborator Author

lbestock commented Dec 3, 2023

He absolutely deletes superfluous relationships, so we and the HM composer have his blessing there.

He also argues against those who would call the section drawing obsolete.

This is like reading the bible if you've been to Sunday school your whole life and imbibed all the stories but never before the original words. I may not have used Harris matrices before, but apparently nearly everything I think and teach about stratigraphy can be found here.

@lbestock
Copy link
Collaborator Author

lbestock commented Dec 3, 2023

Oh, ja, I didn't expect to get so extensive, nor really think about it. Bibby is the author of a chapter in the second Harris book, the one on putting the matrix into practice. I will make us some notes and put the two volumes and the notes in a google folder and remove them from this ticket, once I get a fire going.

@urapadmin
Copy link
Collaborator

@lbestock: FA with only very minor issues:
image

@urapadmin
Copy link
Collaborator

and not anymore substantially wider than the spline version:
image

@urapadmin
Copy link
Collaborator

we also still have wider nodes and I can collapse the empty vertical columns. That'll gain a few more pixels. I think I got it.

@luizaogs
Copy link
Collaborator

I think we can close this one, no?

@urapadmin
Copy link
Collaborator

@lbestock can we close this?

@urapadmin
Copy link
Collaborator

closing it

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kiosk a kiosk issue (not a filemaker issue)
Projects
Status: Done
Development

No branches or pull requests

3 participants