Repository navigation
Release 1.1.0 - #97
Conversation
Story #2022 : Primary and secondary theme mapping
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
| ) | ||
| from chatbot.utils.company_utils import get_company_queryset_for_user, get_user_company, get_user_profile | ||
| from chatbot.utils.knowledge_service.theme_classification_service import ( | ||
| ADMIN_THEME_WAIT_TIMEOUT_SECONDS, ThemeMessageLevel, enqueue_theme_classification, theme_outcome_message, |
There was a problem hiding this comment.
I have kept the max waiting time as 30 seconds for ADMIN_THEME_WAIT_TIMEOUT_SECONDS, after that the theme mapping will happens in background.
rakeshSgr
left a comment
There was a problem hiding this comment.
@vishwanath1004 Please check the comments
| if not pending or response.status_code != 302: | ||
| return response | ||
| task_id, media_name = pending | ||
| outcome = wait_for_theme_classification(task_id, ADMIN_THEME_WAIT_TIMEOUT_SECONDS) |
There was a problem hiding this comment.
Previously, after saving the media, users were immediately redirected to the listing page. With the addition of theme mapping to the existing implementation, we’ve updated the flow so that navigation happens only after the theme mapping is complete. This ensures that users can see the mapped themes immediately on the listing page.
There was a problem hiding this comment.
@vishwanath1004 we can remove this, it may take more, lets it happen in backend
|
|
||
| def get_queryset(self, request): | ||
| return super().get_queryset(request).select_related('created_by').annotate( | ||
| primary_count=Count('primary_media', distinct=True), |
There was a problem hiding this comment.
It adds an number to the each theme, Its a count for each media how many resources are using this as primary theme.
| 'existing_secondary_themes': list(existing_names), | ||
| 'needed': needed, | ||
| 'document': self.document_context(document), | ||
| 'feedback': feedback, |
There was a problem hiding this comment.
No we are not storing in db, feedback used if theme classification is failed any cases we are showing an message.
|
|
||
| logger = logging.getLogger('django') | ||
|
|
||
| THEME_CLASSIFIER_ROUTE = '/theme_classifier' |
There was a problem hiding this comment.
@vishwanath1004 create a constant file theme, and move these constant there
| MEDIA_THEME_FIELDS = ['primary_theme', 'primary_theme_confidence', 'primary_theme_reasoning', 'needs_review'] | ||
| SEARCH_TEXT_MAX_CHARS = 2000 | ||
| THEME_CODE_HASH_LENGTH = 12 | ||
| NEAR_DUPLICATE_NEIGHBOUR_FLOOR = 0.3 |
There was a problem hiding this comment.
we are using that to check the duplicate names of secondary theme.
There was a problem hiding this comment.
thats the minimum matching, if the existing secondary theme name is matching with new one then it will treats as duplicate one so it will not create the secondary theme it will reuse the existing one.
| SEARCH_TEXT_MAX_CHARS = 2000 | ||
| THEME_CODE_HASH_LENGTH = 12 | ||
| NEAR_DUPLICATE_NEIGHBOUR_FLOOR = 0.3 | ||
| NEAR_DUPLICATE_NEIGHBOUR_LIMIT = 10 |
There was a problem hiding this comment.
@vishwanath1004 provide me the brief explanation for these keys
|
|
||
| def acquire_secondary_theme_lock() -> None: | ||
| with connection.cursor() as cursor: | ||
| cursor.execute('SELECT pg_advisory_xact_lock(%s)', [SECONDARY_THEME_LOCK_KEY]) |
There was a problem hiding this comment.
@vishwanath1004 SECONDARY_THEME_LOCK_KEY ? how this works
There was a problem hiding this comment.
This is the ID number of a database lock. The lock stops two classifications running at the same time from both creating the same new secondary theme.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| if not media: | ||
| logger.warning(f"{log_prefix} vector sync skipped: media not found") | ||
| return None | ||
| if media.company_bot and media.company_bot.provider == LLMProvider.OPENAI: |
There was a problem hiding this comment.
@vishwanath1004 we don't need to check the == LLMProvider.OPENAI , check in other places also
| THEME_WAIT_MAX_POLL_SECONDS = 3.0 | ||
| SECONDARY_THEME_MIN_WORDS = 2 | ||
| SECONDARY_THEME_MAX_WORDS = 4 | ||
| MAX_SECONDARY_THEME_COUNT = 10 |
There was a problem hiding this comment.
SECONDARY_THEME_COUNT this is the key to manage the number of secondary themes to assign each document hence this key i have moved to .env from the .bot file. And MAX_SECONDARY_THEME_COUNT is the upper limit for the SECONDARY_THEME_COUNT setting. The setting decides how many secondary themes each document can get (currently 2), and it can never be set above 10.
No description provided.