Skip to content
atfortesPublic

Latest commit

 

History

3 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 

Repository files navigation

ThinkDrag: Semantic Drag-Based Image Editing with Visual Reasoning

Armando Fortes   Tianyi Wei   Chindanai Trakantannarong   Shangchen Zhou   Xingang Pan

S-lab, Nanyang Technological University

Project Page arXiv


ThinkDrag interprets sparse drag handles as semantic edits

It can reason about the intended transformation before generating.

ThinkDrag teaser: for an ambiguous drag on a taxi, a previous method distorts the car, while ThinkDrag reasons that the hood should open; a text prompt can instead redirect the edit to lift the taxi's front.

📖 Overview

ThinkDrag is a drag-based image editing method built on a unified multimodal model. Rather than treating drags as geometric displacements, it interprets them as semantic transformations and can optionally produce a visual reasoning trace before generating the edit. To support this, we construct a training dataset of semantic drag transformations paired with reasoning traces, and introduce DragBench++, a benchmark of challenging drag-based editing scenarios with reference edits.

📮 Update

  • [2026.10] This repo is created.

🚧 TODO

  • Release DragBench++
  • Release Dataset
  • Release Model Weights
  • Release Inference Code

🖼️ Results

Semantic transformations. Same input and drag handles across drag-based image editing methods.

Qualitative comparison on DragBench and DragBench++: DragDiffusion, DragLoRA, GeoDrag and LightningDrag mostly shift or distort pixels, while ThinkDrag turns the dog's head, spreads the parrot's wings, raises the blinds and turns the hippo's head.

Reasoning-guided edits. ThinkDrag can first infer the intended edit from the handles, then use that reasoning trace to guide generation.

Direct generation versus reasoning mode: ThinkDrag reasons that the drags should open the refrigerator door, make the painted waves taller and cross the man's arms, and the reasoning-mode outputs realize that intent.

🧩 Method

Dataset. Synthetic image pairs are filtered, matched with semantic keypoints, and paired with reasoning chains.

ThinkDrag dataset pipeline: synthesized image pairs are filtered for plausibility, matched with semantic keypoints to form drag instructions, and paired with reasoning traces.

Model. Drags are represented as structured text tokens and Drag Endpoint Markers in a unified multimodal model.

ThinkDrag model overview: a unified multimodal model takes the image and drag instruction, predicts a reasoning trace by next-token prediction and the edited image by velocity prediction, guided by Drag Endpoint Markers.

📑 Citation

If you find our work useful, please consider giving this repo a ⭐ to support our work and citing the following paper:

@article{fortes2026thinkdrag,
  title   = {ThinkDrag: Semantic Drag-Based Image Editing with Visual Reasoning},
  author  = {Fortes, Armando and Wei, Tianyi and Trakantannarong, Chindanai and Zhou, Shangchen and Pan, Xingang},
  journal = {arXiv preprint arXiv:2610.13316},
  year    = {2026}
}

©️ License

This project is licensed under NTU S-Lab License 1.0. Redistribution and use should follow this license.

🤝 Acknowledgements

We would like to thank the drag-based editing community and the many works that paved the way for this one since Drag Your GAN. In particular, we compare against DragDiffusion, DragNoise, DragLoRA, FastDrag, GeoDrag, InstantDrag & LightningDrag.

We also thank the following projects that made this work possible:

  • BAGEL is the base unified multimodal model that ThinkDrag is built upon.
  • Qwen3-VL is used to write source-image and edit prompts for the dataset.
  • Qwen-Image, FLUX.2 & Z-Image are used to synthesize the dataset image pairs.
  • Gemini 3 is used to estimate semantic keypoints and write reasoning traces.

Releases

Contributors