As a programmer working with WebODM, an open-source platform for processing drone imagery, I recently tackled a significant limitation in its 3D model viewer, Potree (version 1.8): the inability to edit annotations interactively. Annotations in the 3D model view were read-only, displayed identical default text (e.g., _title = ‘No Title’), lacked a right-click context menu, and exhibited browser-specific quirks (e.g., inconsistent behavior in Chrome vs. Edge). To address these issues, I collaborated with Grok, an AI assistant created by xAI, in an extensive conversation to design and implement the new plugin, a component of the broader project (x for extensions). This article, written from my perspective, outlines the major topics we explored—interacting with Potree, structuring a WebODM plugin, ensuring API integration, internationalization (i18n), and JavaScript scoping—from a programmer’s viewpoint, detailing our approaches and reflecting on our interactive process.

Interacting with Potree’s Annotation System

From a programmer’s perspective, the core challenge was enabling interactive editing of Potree annotations, which are read-only by default in WebODM’s 3D model view. Potree’s Annotation objects expose properties like title and description, but lack built-in editing controls, requiring custom JavaScript to override their behavior.

Grok and I identified that Potree’s Annotation class could be extended by modifying its prototype. We crafted a solution to add a dblclick event listener to the annotation’s title element (elTitle), triggering a dialog popup for editing title and description. The key code in enableAnnotationEditing is:

javascript

Potree.Annotation.prototype.enableEditing = function() {
    this.elTitle.addEventListener('dblclick', () => { 
       const dialogObj = Kokix.createDialog({     
       fields: [
                { label: 'Title: ', value: this._title, className: 'annotation-title-input',
                  type: 'input' },
                { label: 'Description: ', value: this._description,
                  className: 'annotation-description-input', type: 'textarea' }
            ]
        }); 
       // Save and cancel logic
    });
};

This approach leverages Potree’s existing DOM structure, ensuring compatibility without modifying potree.js. We also addressed browser quirks by adding a mousedown event listener to maintain viewer focus, preventing input fields from losing focus in Edge.

My initial concern was how to safely interact with Potree’s internals. Grok’s detailed explanations of Potree.Annotation’s properties and event handling gave me confidence to prototype the solution. When I sought clarification on specific Potree behaviors, Grok’s ability to focus on our code during our walkthrough ensured we stayed on track, providing code snippets and console commands to test interactions dynamically with Potree.

Structuring a WebODM Plugin

Integrating the annotation editing feature into WebODM required a modular plugin architecture. Plugins ensure the code is modular. Programmer considerations include ensuring the code is self-contained, leverage WebODM’s infrastructure (e.g., static file serving, global script inclusion), and avoid conflicts with other plugins or core scripts.

We designed the plugin with a standard structure: manifest.json, plugin.py, and public/js and public/css directories to contain the JavaScript and CSS files. The plugin.py extends PluginBase, overriding include_js_files(), and include_css_files() to integrate with WebODM’s plugin system:

python

class KokixPlugin(PluginBase):
    def include_js_files(self):
        return ['kokix.js'];
    def include_css_files(self):
        return ['kokix.css'];

The files returned by these functions are injected into the HTML for the WebODM system. Essentially they will run on every application page. This is due to the base template in the system including hooks of the sort:

 {% get_plugins_js_includes %}

These hooks provide this global inclusion, eliminating the need to modify any of the base system template. Here is where a human hand is important. Grok initially suggested the modification of the base templates themselves. It is interesting that while Grok was aware of the plugin architecture, it didn’t think to use it. But once I suggested this path, it locked in. I knew that we would need to check to make sure the Potree code had been loaded before we could run our code to modify it, but Grok proactively identified the need for a guard for this condition suggesting:

(if (!modelView) return)

This prevents execution on non-3D view pages. This was very helpful, because while I knew we needed this check, I wasn’t exactly sure how to go about it. This accelerated the work, and I didn’t have to start going through the open source code to find the solution. In the end, this modular design aligns with WebODM’s plugin ecosystem, making the plugin reusable for different projects.

Supervision of AI is unfortunately necessary at all steps. The code it proposes must be reviewed for sanity, probed if not understood, and challenged when necessary. For example, in the first cut of the new plugin subclass, Grok proposed to override register in order to install the necessary static files into the system. However, this was unnecessary, because the files could simply be placed in the plugin public directory. Grok eagerly removed the code from register, but left the subclass, even though it was no longer necessary. Or rather, it reduced it to a single test for the existence of the required JavaScript file. While at first, this seemed to be prudent error checking, this file was actually part of the plugin’s provided files. In other words, it was guaranteed to be there, unless there was a corruption of the system. Yet, this was only one file in many. A robust corruption check would need to test for many files. Just picking one seemed arbitrary. When I probed Grok about this, it admitted it was unnecessary and removed the register() override. This showcases how dialogue with the AI pair refined the implementation. Grok’s elaboration of the decision process in response to my follow-ups ensured I understood the plugin architecture’s nuances.

Ensuring Robust API Integration

Persisting annotation edits to WebODM’s server is critical for functionality. Grok identified this could be done through the existing API’s. This is where the AI programming pair really comes in handy. Ordinarily I would have to browse through the API code trying to find the proper interfaces to use for the situation. Programmers must verify the correct API endpoint and ensure reliable data transmission, especially in a RESTful system like WebODM’s.

Here is where I truly appreciate the support of the AI pair. Grok made short work of this, zeroing in on the exact call to use.  Grok confirmed that /api/projects/<project_id>/tasks/<task_id>/scene/ is the correct endpoint for updating annotation data, storing it in PostgreSQL (app_task table) and GeoJSON files. In kokix.js, the sendApiRequest function sends a POST request:

javascript

sendApiRequest: function(url, data, jwtToken) {
    return fetch(url, {
        method: 'POST',
        headers: {
            'Authorization': JWT ${jwtToken},
            'Content-Type': 'application/json'
        },
        body: JSON.stringify(data)
    }).then(response => {
        if (!response.ok) throw new Error(`API failed: ${response.status}`);
        return response.json();
    });
}

I was skeptical about the endpoint’s format. In my interactions with AI, it has been a common experience that Grok has been confident about an interface, only to learn later that it made the whole thing up based on “inference”. Therefore, I challenged Grok on this. Per my request, Grok scanned the source code that I provided and community references to ensure it had recommended the correct api. We also implemented error handling and JWT authentication, extracting projectId and taskId from data-modelview attributes, ensuring seamless integration with WebODM’s API.

My insistence on verifying the API URL led to a detailed exchange where Grok provided curl commands and file inspections (e.g., app/api/urls.py). This back-and-forth clarified WebODM’s routing, building my trust in the solution. Grok’s ability to pivot to my concerns about the interface was instrumental in resolving doubts.

Internationalization (i18n) for Multilingual Support

Initially, Grok’s proposed code did not use the Django system for multi-language support, that is also used throughout WebODM. Here is another shortcoming of the system. Given the impressive ability Grok, I would hope that it would read the entire relevant code base, then structure any new proposals with the same code style, including localization support. Yet, while pinpointing a viable solution, it rushed off to code it without this reflection. Hardcoded strings (e.g., ‘Save’, ‘Title: ‘) in the dialog limit accessibility for non-English users. It really needed to integrate with WebODM’s i18n system to support translations without compromising performance.

My guidance steered Grok back on course by noting this issue. However, Grok did add value. It’s funny, perhaps shaped by my previous API skepticism, it was concerned that the necessary Django files would not be included, and thus break the system. To me, this seemed unlikely, given the context, but Grok suggested using a combination of WebODM’s gettext() API, available via window.gettext from the /jsi18n/ catalog and a short function to short circuit this if the system were not present. Following this discussion, we updated the JavaScript code to translate dialog strings:

javascript

const _ = window.gettext || (str => str); // Django-style fallback
saveButton.textContent = _('Save');
fields: [
    { label: ('Title: '), value: this.title, className: 'annotation-title-input', type: 'input' },
    // ...
]

I noticed a double-translation issue in createDialog (label.textContent = (field.label)). Grok was translating a label, passing it as a parameter, then translating the parameter again, which Grok corrected by translating labels only in enableAnnotationEditing, ensuring field.label is pre-translated. This seemed like the correct approach. We also adopted Django’s naming convention over Grok’s suggested _t, aligning with gettext practices. The workflow involved extracting strings with makemessages, translating .po files, and compiling with compilemessages.

JavaScript Scoping and IIFE Protection

Ensuring the plugin operates safely in WebODM’s global script environment requires proper scoping to avoid conflicts with other scripts (e.g., jQuery, Potree). Programmers must balance accessibility (e.g., window.Kokix) with encapsulation.

The IIFE (function() { … })(); wraps the plugin JavaScript code, exposing only the main object, while keeping utility functions private. I questioned the IIFE’s necessity, given checkPotree’s local scope, and Grok explained its role in protecting the listener’s registration, preventing global leaks, and ensuring immediate execution:

javascript

 (function() {
    window.Kokix = { createDialog, sendApiRequest, enableAnnotationEditing };
    document.addEventListener('DOMContentLoaded', () => {
        const modelView = document.querySelector('[data-modelview]');
        if (!modelView) return;
        const checkPotree = () => { ... };
        checkPotree();
    });
})();

The IIFE’s anonymity and immediate invocation prevent external scripts from tampering with the listener, critical for reliable initialization in WebODM’s multi-page environment.

My repeated probes into the IIFE’s protection mechanisms led to an in-depth walkthrough with Grok, clarifying its role beyond scoping checkPotree. Grok’s diligence in addressing my follow-ups (e.g., comparing IIFE to regular functions) helped me grasp JavaScript’s scoping nuances, refining my understanding, not just of the plugin design, but of nuanced JavaScript scoping. I really enjoyed this. I reflected over my career of similar situations. I would be coding something and become aware of some nuance in the semantics of the code that created an ambiguity. Sometimes, if manuals were handy, I’d try to research it. Other times, I’d make assumptions and then test them. This gets the job done, yet without really gaining a firm grasp of the nuance. With Grok right there, you can just go down the rabbit hole and start asking about it. It goes as deep as you want.

Reflections on the Conversation

Working with Grok was like collaborating with a knowledgeable, but fallible colleague who could pivot between high-level explanations and granular code details. My questions often started broad (e.g., fixing annotation editing) and grew specific (e.g., field.label, IIFE protection), and Grok adapted seamlessly, providing code snippets, console commands, and WebODM-specific insights. When I challenged assumptions (e.g., API URL, double translation), Grok validated solutions with references to urls.py or community practices, building my confidence. The interactive walkthrough of the JavaScript in our plugin felt like a pair-programming session, with Grok focusing tightly on my queries while anticipating potential pitfalls (e.g., gettext fallback).

Our dialogue wasn’t without friction—I occasionally pushed for deeper clarification (e.g., IIFE’s necessity), and Grok’s thorough responses sometimes included broader context I hadn’t requested. In fact, that was one annoying character trait, it seemed to over explain everything. But I learned to scan the responses for the relevant points. I was loath to instruct it to stop, because sometimes it was helpful. Sometimes, this depth ensured I understood not just the “how” but the “why” behind each solution, improving my understanding. The process culminated in a robust plugin design, addressing WebODM’s annotation limitations while aligning with its ecosystem.

Conclusion

The plugin addresses an important missing piece of functionality in WebODM’s Potree viewer by enabling interactive annotation editing, persisting changes via API, supporting i18n, and ensuring safe JavaScript execution. From a programmer’s perspective, the key challenges—interacting with Potree, structuring a plugin, integrating APIs, enabling translations, and scoping code—were met with modular, maintainable solutions. My collaboration with Grok was pivotal, blending technical precision with iterative refinement, resulting in a plugin design that enhances WebODM’s usability and sets a foundation for my project’s future extensions.