Merge branch 'master' of github.com:hanhainebula/FlagEmbedding

research/llm_dense_retriever/examples/bge-en-icl/MTEB/AmazonCounterfactualClassification.json

@@ -1 +1,14 @@
-[{"query": "I wish I could have used this head set but the day I received it it wouldn't even turn on and I really wanted this product to work I'm very disappointed.", "response": "I wish I could have used this head set but the day I received it it wouldn't even turn on and I really wanted this product to work I'm very disappointed."}, {"query": "I would advise that instead of trying to follow these poor instructions, Google it.", "response": "I would advise that instead of trying to follow these poor instructions, Google it."}, {"query": "I wrote to Monster customer service before ordering and they told me it would be fine to use without a converter and it was absolutely true.", "response": "I wrote to Monster customer service before ordering and they told me it would be fine to use without a converter and it was absolutely true."}]
+[
+    {
+        "query": "I wish I could have used this head set but the day I received it it wouldn't even turn on and I really wanted this product to work I'm very disappointed.",
+        "response": "counterfactual"
+    },
+    {
+        "query": "I would advise that instead of trying to follow these poor instructions, Google it.",
+        "response": "not-counterfactual"
+    },
+    {
+        "query": "I wrote to Monster customer service before ordering and they told me it would be fine to use without a converter and it was absolutely true.",
+        "response": "not-counterfactual"
+    }
+]

research/llm_dense_retriever/examples/bge-en-icl/MTEB/AmazonPolarityClassification.json

@@ -1 +1,18 @@
-[{"query": "Hunting the Hard Way Thia was a gift for my Husband, who loved the book. It arrived on the date we were told it would.", "response": "Hunting the Hard Way Thia was a gift for my Husband, who loved the book. It arrived on the date we were told it would."}, {"query": "Poor DVD Has too many interviews with people at the Live THomas day in Penn. My kids were annoyed and hated this DVD.", "response": "Poor DVD Has too many interviews with people at the Live THomas day in Penn. My kids were annoyed and hated this DVD."}, {"query": "Ludicrous and silly I remember getting this book so faintly that that says alot about my opinion of it. Basically, while I will entertain lots of odd ideas and theories, this book was basically silly.", "response": "Ludicrous and silly I remember getting this book so faintly that that says alot about my opinion of it. Basically, while I will entertain lots of odd ideas and theories, this book was basically silly."}, {"query": "Artistry I think that the Deodato concerts are very rich, as he used real strings and band musicians, as well as you can appreciate the John Tropea excelent renditions on guitar.", "response": "Artistry I think that the Deodato concerts are very rich, as he used real strings and band musicians, as well as you can appreciate the John Tropea excelent renditions on guitar."}]
+[
+    {
+        "query": "Hunting the Hard Way Thia was a gift for my Husband, who loved the book. It arrived on the date we were told it would.",
+        "response": "positive"
+    },
+    {
+        "query": "Poor DVD Has too many interviews with people at the Live THomas day in Penn. My kids were annoyed and hated this DVD.",
+        "response": "negative"
+    },
+    {
+        "query": "Ludicrous and silly I remember getting this book so faintly that that says alot about my opinion of it. Basically, while I will entertain lots of odd ideas and theories, this book was basically silly.",
+        "response": "negative"
+    },
+    {
+        "query": "Artistry I think that the Deodato concerts are very rich, as he used real strings and band musicians, as well as you can appreciate the John Tropea excelent renditions on guitar.",
+        "response": "positive"
+    }
+]

research/llm_dense_retriever/examples/bge-en-icl/MTEB/AmazonReviewsClassification.json

@@ -1 +1,22 @@
-[{"query": "DO NOT ORDER THIS\\n\\nThis isn't what's described at all. Taking it out of the package lace was cut upon arrival, wig was cut to like 14 inch, not curly, and smelled like cigarettes. I obviously was sent what someone returned, disgusting.Not what I ordered at all, not pleased at all. I want my money back DO NOT ORDER", "response": "DO NOT ORDER THIS\\n\\nThis isn't what's described at all. Taking it out of the package lace was cut upon arrival, wig was cut to like 14 inch, not curly, and smelled like cigarettes. I obviously was sent what someone returned, disgusting.Not what I ordered at all, not pleased at all. I want my money back DO NOT ORDER"}, {"query": "And I can\u2019t return it\\n\\nThis product seemed like good quality but it does not stay stuck to the soles at all. You walk a few steps and then you find the black shoe grip somewhere on the floor.", "response": "And I can\u2019t return it\\n\\nThis product seemed like good quality but it does not stay stuck to the soles at all. You walk a few steps and then you find the black shoe grip somewhere on the floor."}, {"query": "Three Stars\\n\\nnew yearly subscription plan is horrible, but the product still works as it did in the past", "response": "Three Stars\\n\\nnew yearly subscription plan is horrible, but the product still works as it did in the past"}, {"query": "I like how it has lots of pockets to put stuff ...\\n\\nI like how it has lots of pockets to put stuff in. I would have liked to have a shorter securing strap so it would not slide around so much. Good product.", "response": "I like how it has lots of pockets to put stuff ...\\n\\nI like how it has lots of pockets to put stuff in. I would have liked to have a shorter securing strap so it would not slide around so much. Good product."}, {"query": "Great\\n\\nIt is really good. That's my favorite. THANK YOU", "response": "Great\\n\\nIt is really good. That's my favorite. THANK YOU"}]
+[
+    {
+        "query": "DO NOT ORDER THIS\\n\\nThis isn't what's described at all. Taking it out of the package lace was cut upon arrival, wig was cut to like 14 inch, not curly, and smelled like cigarettes. I obviously was sent what someone returned, disgusting.Not what I ordered at all, not pleased at all. I want my money back DO NOT ORDER",
+        "response": "1 star"
+    },
+    {
+        "query": "And I can\u2019t return it\\n\\nThis product seemed like good quality but it does not stay stuck to the soles at all. You walk a few steps and then you find the black shoe grip somewhere on the floor.",
+        "response": "2 star"
+    },
+    {
+        "query": "Three Stars\\n\\nnew yearly subscription plan is horrible, but the product still works as it did in the past",
+        "response": "3 star"
+    },
+    {
+        "query": "I like how it has lots of pockets to put stuff ...\\n\\nI like how it has lots of pockets to put stuff in. I would have liked to have a shorter securing strap so it would not slide around so much. Good product.",
+        "response": "4 star"
+    },
+    {
+        "query": "Great\\n\\nIt is really good. That's my favorite. THANK YOU",
+        "response": "5 star"
+    }
+]

research/llm_dense_retriever/examples/bge-en-icl/MTEB/ArguAna.json

@@ -1 +1,14 @@
-[{"query": "People will die if we don\u2019t do animal testing Every year, 23 new drugs are introduced in the UK alone.[13] Almost all will be tested on animals. A new drug will be used for a long time. Think of all the people saved by the use of penicillin. If drugs cost more to test, that means drug companies will develop less. This means more people suffering and dying", "response": "People will die if we don\u2019t do animal testing Every year, 23 new drugs are introduced in the UK alone.[13] Almost all will be tested on animals. A new drug will be used for a long time. Think of all the people saved by the use of penicillin. If drugs cost more to test, that means drug companies will develop less. This means more people suffering and dying"}, {"query": "Survival of the fittest It is natural for human beings to farm, kill, and eat other species. In the wild there is a brutal struggle for existence as is shown by Darwin\u2019s On the Origin of the Species. The fact that we humans have succeeded in that struggle by exploiting our natural environment means that we have a natural right over lower species. The concept of survival of the fittest may seem outdated but it is still the defining order of nature. In fact farming animals is much less brutal than the pain and hardship that animals inflict on each other naturally in the wild.", "response": "Survival of the fittest It is natural for human beings to farm, kill, and eat other species. In the wild there is a brutal struggle for existence as is shown by Darwin\u2019s On the Origin of the Species. The fact that we humans have succeeded in that struggle by exploiting our natural environment means that we have a natural right over lower species. The concept of survival of the fittest may seem outdated but it is still the defining order of nature. In fact farming animals is much less brutal than the pain and hardship that animals inflict on each other naturally in the wild."}, {"query": "Underground Nuclear Storage is Expensive. Underground nuclear storage is expensive. This is because the deep geological repositories needed to deal with such waste are difficult to construct. This is because said repositories need to be 300m underground and also need failsafe systems so that they can be sealed off should there be a leak. For smaller countries, implementing this idea is almost completely impossible. Further, the maintenance of the facilities also requires a lot of long-term investment as the structural integrity of the facilities must consistently be monitored and maintained so that if there is a leak, the relevant authorities can be informed quickly and efficiently. This is seen with the Yucca mountain waste repository site which has cost billions of dollars since the 1990s and was eventually halted due to public fears about nuclear safety.", "response": "Underground Nuclear Storage is Expensive. Underground nuclear storage is expensive. This is because the deep geological repositories needed to deal with such waste are difficult to construct. This is because said repositories need to be 300m underground and also need failsafe systems so that they can be sealed off should there be a leak. For smaller countries, implementing this idea is almost completely impossible. Further, the maintenance of the facilities also requires a lot of long-term investment as the structural integrity of the facilities must consistently be monitored and maintained so that if there is a leak, the relevant authorities can be informed quickly and efficiently. This is seen with the Yucca mountain waste repository site which has cost billions of dollars since the 1990s and was eventually halted due to public fears about nuclear safety."}]
+[
+    {
+        "query": "People will die if we don\u2019t do animal testing Every year, 23 new drugs are introduced in the UK alone.[13] Almost all will be tested on animals. A new drug will be used for a long time. Think of all the people saved by the use of penicillin. If drugs cost more to test, that means drug companies will develop less. This means more people suffering and dying",
+        "response": "animals science science general ban animal testing junior Many of these drugs are \u201cme too\u201d drugs \u2013 ones with a slight change that doesn\u2019t make much difference to an existing drug. [14] So often the benefits from animal testing are marginal, and even if there was a slight increase in human suffering, it would be worth it based on the animal suffering saved."
+    },
+    {
+        "query": "Survival of the fittest It is natural for human beings to farm, kill, and eat other species. In the wild there is a brutal struggle for existence as is shown by Darwin\u2019s On the Origin of the Species. The fact that we humans have succeeded in that struggle by exploiting our natural environment means that we have a natural right over lower species. The concept of survival of the fittest may seem outdated but it is still the defining order of nature. In fact farming animals is much less brutal than the pain and hardship that animals inflict on each other naturally in the wild.",
+        "response": "The claim of human entitlement over other species based on 'survival of the fittest' is flawed. While Darwin's theory highlights competition, it doesn't justify exploitation. Our capacity for empathy and moral reasoning surpasses mere survival instincts. Farming still inflicts suffering, contradicting the notion of human superiority. Ethical considerations should guide our treatment of animals, not outdated notions of natural selection."
+    },
+    {
+        "query": "Underground Nuclear Storage is Expensive. Underground nuclear storage is expensive. This is because the deep geological repositories needed to deal with such waste are difficult to construct. This is because said repositories need to be 300m underground and also need failsafe systems so that they can be sealed off should there be a leak. For smaller countries, implementing this idea is almost completely impossible. Further, the maintenance of the facilities also requires a lot of long-term investment as the structural integrity of the facilities must consistently be monitored and maintained so that if there is a leak, the relevant authorities can be informed quickly and efficiently. This is seen with the Yucca mountain waste repository site which has cost billions of dollars since the 1990s and was eventually halted due to public fears about nuclear safety.",
+        "response": "While initial construction and maintenance entail significant costs, advancements in technology offer more cost-effective solutions. Modular storage designs and improved monitoring systems mitigate expenses. Collaborative international efforts can also distribute costs. Additionally, public concerns can be addressed through transparent safety protocols and community engagement, ensuring responsible nuclear waste management without exorbitant expenditure. Underground nuclear storage isn't inherently prohibitive."
+    }
+]