Patch: deduplicate add objects (#714)

* update reader and search strategy

* set strategy reader and search config

* fix install problem

* fix

* fix test

* turn off graph recall

* turn off graph recall

* turn off graph recall

* fix Searcher input bug

* fix Searcher

* fix Search

* fix bug

* adjust strategy reader

* adjust strategy reader

* adjust search config input

* reformat code

* re pr

* format repair

* fix time issue

* develop feedback process

* feedback handler configuration

* upgrade feedback using

* add threshold

* update prompt

* update prompt

* fix handler

* add feedback scheduler

* add handler change node update

* add handler change node update

* add handler change node update

* add handler change node update

* fix interface input

* add chunk and ratio filter

* update stopwords

* fix messages queue

* add seach_by_keywords_LIKE

* add doc filter

* add retrieve query

* add retrieve queies

* patch info filter

* add log and make embedding safety net

* add log and make embedding safety net

* deduplicate add objects

---------

Co-authored-by: 黑布林 <[email protected]>
Co-authored-by: CaralHsi <[email protected]>
Co-authored-by: chunyu li <[email protected]>

Author: 4 people

src/memos/mem_feedback/feedback.py

@@ -618,7 +618,19 @@ def correct_item(data):
             return None
 
         dehallu_res = [correct_item(item) for item in operations]
-        llm_operations = [item for item in dehallu_res if item]
+        dehalluded_operations = [item for item in dehallu_res if item]
+
+        # deduplicate add objects
+        add_texts = []
+        llm_operations = []
+        for item in dehalluded_operations:
+            if item["operation"].lower() == "add" and "text" in item and item["text"]:
+                if item["text"] in add_texts:
+                    continue
+                llm_operations.append(item)
+                add_texts.append(item["text"])
+            elif item["operation"].lower() == "update":
+                llm_operations.append(item)
 
         # Update takes precedence over add
         has_update = any(item.get("operation").lower() == "update" for item in llm_operations)