Kangbo Lu

🤖 Interpreting the decision making process of RL agents can be tricky. How about training them to tell us what data matters influences the decision most? “Self-Supervised Discovering of Interpretable Features for Reinforcement Learning” by Wenjie Shi, Gao Huang, Shiji Song, Zhuoyuan "Jacob" Wang, Tingyu Lin, and Cheng Wu from Tsinghua University brings interpretable RL to Duckietown using a framework powered by self-supervised attention mechanisms. In other words, the authors built a tiny network (#SSINet) that sits in front of any pre-trained vision policy, and learns a mask that keeps only the pixels the policy must see. This model (a) learns where to "look" in its camera input while driving, (b) identifies which pixels influence its steering decisions, and (c) explains both successful and failed maneuvers. Without human labels nor hand-crafted features, this contribution can help in debugging, teaching, and building trust in autonomous systems! Learn more: https://hubs.la/Q03zh9yB0 🔔 Follow us at Duckietown Robotics to learn about how robot autonomy is changing the world. #InterpretableRL #AIRobotics #ReinforcementLearning #Perception #ComputerVision #EmbodiedAI #RobotAutonomy

51

1 Comment

We all know Neural Networks are black box models So now what is this ? Visualizing Neural Networks - just woaooo At Design Korea 2024, Kim Seonghyun presented a real-time interactive visualization of a neural network trained on MNIST using PyTorch — visualized through Processing and rendered with OpenGL instancing. You can literally watch the data flow: → 784-pixel input → Layer-wise transformation → Flattened vector → Softmax output across 0–9 #AI #DeepLearning #Explainability #NeuralNetworks #AIxArt

6

ProRAG: Process-Supervised Reinforcement Learning for Retrieval-Augmented Generation Link to paper: https://lnkd.in/giJ67ms4 Authors Zhao Wang, Ziliang Zhao, Zhicheng Dou Abstract Reinforcement learning (RL) has become a promising paradigm for optimizing Retrieval-Augmented Generation (RAG) in complex reasoning tasks. However, traditional outcome-based RL approaches often suffer from reward sparsity and inefficient credit assignment, as coarse-grained scalar rewards fail to identify specific erroneous steps within long-horizon trajectories. This ambiguity frequently leads to "process hallucinations", where models reach correct answers through flawed logic or redundant retrieval steps. Although recent process-aware approaches attempt to mitigate this via static preference learning or heuristic reward shaping, they often lack the on-policy exploration capabilities required to decouple step-level credit from global outcomes. To address these challenges, we propose ProRAG, a process-supervised reinforcement learning framework designed to integrate learned step-level supervision into the online optimization loop. Our framework consists of four stages: (1) Supervised Policy Warmup to initialize the model with a structured reasoning format; (2) construction of an MCTS-based Process Reward Model (PRM) to quantify intermediate reasoning quality; (3) PRM-Guided Reasoning Refinement to align the policy with fine-grained process preferences; and (4) Process-Supervised Reinforcement Learning with a dual-granularity advantage mechanism. By aggregating step-level process rewards with global outcome signals, ProRAG provides precise feedback for every action. Extensive experiments on five multi-hop reasoning benchmarks demonstrate that ProRAG achieves superior overall performance compared to strong outcome-based and process-aware RL baselines, particularly on complex long-horizon tasks, validating the effectiveness of fine-grained process supervision. Link to paper: https://lnkd.in/giJ67ms4 #rag #research #rl #supervisedrl

16

3 Comments

When I started building Ron, I wasn’t trying to create “just another BI tool.” I’ve tested enough dashboards, tried enough no-code platforms, and fine-tuned enough prompts to know one thing: ❗️Most data tools look powerful on the surface — but in reality, they still expect you to do the heavy lifting. You need to: • Know what to ask • Know how to clean data • Know how to build dashboards that actually tell a story But here’s the truth: most teams don’t have time for that. And they shouldn’t need to. So I took a different path. Not “training people to use AI,” but “training AI to understand people.” That’s how Ron came to life — an AI teammate for Business Intelligence. Not just a data assistant, but a thinking partner who speaks the language of goals, KPIs, and clarity. With just a few inputs, Ron helps you: ✅ Turn datasets into clean, usable reports ✅ Choose the right charts to communicate clearly ✅ Analyze performance across teams ✅ Summarize insights in plain English ✅ And deliver reporting that execs actually read No complex prompts. No technical setup. Just results — fast, clear, and ready to act on. With Ron, you don’t need to be a BI expert. You just need to know your business. Ron takes care of the rest. Curious how it works? Check out it here for 3 days trials: https://ai-mentor.io/

4

🚀 Movie Sentiment Classification Using Simple RNN 🚀 This project predicts the sentiment of movie reviews using a Simple Recurrent Neural Network (SimpleRNN). Sentiment classification helps understand audience reactions and can be applied to movie recommendations, review analysis, and social media monitoring. Dataset: Sentiment (Text Form), Target: (1 = Positive, 0 = Negative). Git Hub Link (Demo Attached in Readme): https://lnkd.in/gDBscZ2S

10

1 Comment

Shuai Guo explains how LangGraph can be used to build deep research agents. He breaks down the fundamentals of Google’s open-source full-stack implementation and shows how to handle workflows, state management, and guardrails effectively.

4

💥💥💥 Deep Research Agents: A Systematic Examination And Roadmap Yuxuan Huang, Yihang Chen, Haozheng Zhang, Kang Li, Meng Fang, Linyi Yang, Xiaoguang Li, Lifeng Shang, Songcen Xu, Jianye Hao, Kun Shao, Jun Wang Abstract The rapid progress of Large Language Models (LLMs) has given rise to a new category of autonomous AI systems, referred to as Deep Research (DR) agents. These agents are designed to tackle complex, multi-turn informational research tasks by leveraging a combination of dynamic reasoning, adaptive long-horizon planning, multi-hop information retrieval, iterative tool use, and the generation of structured analytical reports. In this paper, we conduct a detailed analysis of the foundational technologies and architectural components that constitute Deep Research agents. We begin by reviewing information acquisition strategies, contrasting API-based retrieval methods with browser-based exploration. We then examine modular tool-use frameworks, including code execution, multimodal input processing, and the integration of Model Context Protocols (MCPs) to support extensibility and ecosystem development. To systematize existing approaches, we propose a taxonomy that differentiates between static and dynamic workflows, and we classify agent architectures based on planning strategies and agent composition, including single-agent and multi-agent configurations. We also provide a critical evaluation of current benchmarks, highlighting key limitations such as restricted access to external knowledge, sequential execution inefficiencies, and misalignment between evaluation metrics and the practical objectives of DR agents. Finally, we outline open challenges and promising directions for future research. 👉 https://lnkd.in/dTr6iGxw A curated and continuously updated repository of DR agent research is available 👉 https://lnkd.in/dbcndduk #machinelearning

4

sktime 0.38.3, pytorch-forecasting 1.4.0 have been released! Highlights: - new hierarchical reconciliation module, non-negative reconciliation (Felipe Angelim) - Beta 1 for pytorch-forecasting 2.0 API - feedback appreciated! (Aryan Saini, Pranav Bhat, Andrea Gobbi, Sandeep Kumar Kushwaha, Franz Király) - benchmarking module for time series classification (Jigyasu Krishnan‎) - momentfm foundation model for forecasting and time series classification (Julian Fong) - TimeXer interface in pytorch-forecasting (Pranav Bhat) - ForecastKnownValues can now handle hierarchical data (Dr. Robert Kübler) - Sliding greedy splitter for time series splitting (Marc Rovira) - forecasting evaluate / benchmark utility can now handle global splits (Benedikt Heidrich) - skforecast recursive reducer interface now also supports in-sample quantile predictions (LOCHAN PAUDEL) Changelogs: sktime - https://lnkd.in/etekGzZ9 pytorch-forecasting - https://lnkd.in/efvVffbT #sktime #opensource #python #timeseries #ai #ml #datascience

82

3 Comments

Like recall and precision metrics in ML, we have been diving into fidelity and privacy metrics for synthetic data generation. As we explore privacy-safe data solutions at Rockfish, understanding these trade-offs is becoming crucial. If you're curious about the technical details, this is a must-read!

6

Creating Jobs. Connecting People. Digireha launches a new side-job talent matching service for rehabilitation professionals in Japan! As demand for PTs, OTs, and STs grows across after-school and early intervention facilities, we’re building a bridge between: Experts who want flexible work. × Facilities that urgently need specialised support. Driven by our mission — Creating a future not defined by disability, we’re empowering professionals and expanding possibilities for every child. #Digireha #RehabJobs #Inclusion #PT #OT #ST #ChildDevelopment #Japan

5

New short course: DSPy: Build and Optimize Agentic Apps DSPy is a powerful open-source framework for automatically tuning prompts for GenAI applications. In this course, you'll learn to use DSPy, together with MLflow. This is built in partnership with Databricks and taught by Chen Qian, co-lead of the DSPy framework. Many AI builders spend hours hand-tuning prompts. When given a set of evals, DSPy automates this process. It’s especially useful for optimizing prompts, including few-shot prompts, in complex agentic AI workflows. Further, if you switch an application to a newer LLM, performance can degrade if your prompts were optimized to the previous model. DSPy automatically optimizes the entire system for the new LLM as well, using just a few evaluation examples. This course teaches DSPy works, and best practices for using it. You’ll write programs using DSPy’s signature-based programming model, debug them with MLflow tracing -- to gain visibility into how different parts of a pipeline, as well as how the overall system, are performing -- and automatically improve their accuracy with DSPy Optimizer. Please sign up here: https://lnkd.in/gdjae8AX

4,671

190 Comments

AutoGen (now AG2) is an open-source multi-agent conversation framework for building and orchestrating AI agents. It provides a set of classes and functions for defining agents that can communicate (via messages), perform actions, and collaborate on tasks. Key components include Agent protocols, concrete agent classes (e.g. AssistantAgent, UserProxyAgent), and chat orchestration tools (GroupChat, initiate_chats, run_swarm, etc.).

14

Researchers at Renmin University of China and Xiaohongshu have released a study on Deep Reasoning Agents with Scalable Toolsets My Honest Analysis: 1. This work presents a unified agent framework that enables reasoning models to autonomously think, discover tools, and execute actions in one continuous process. 2. It breaks away from traditional fixed “Reason-Act-Observe” loops, allowing the agent to dynamically select tools rather than rely on pre-defined workflows. 3. A key innovation is the autonomous memory folding mechanism, compressing interaction histories into structured episodic, working, and tool memories, improving efficiency and stability. 4. The paper introduces ToolPO, an end-to-end reinforcement learning strategy that assigns fine‐grained credit to tool-invocation tokens via advantage attribution. 5. Experiments across eight benchmarks—both general tool-use tasks and downstream applications—show DeepAgent consistently outperforms strong baselines. 6. Notably, in open-set tool retrieval scenarios (where tools aren’t pre-given) the agent maintains robustness and scalability. 7. The brain-inspired memory schema (episodic + working + tool memory) helps the agent “take a breath” and restructure its thinking when stuck. arXiv 8. This research advances the frontier of general-purpose autonomous agents that can handle real-world tasks with large, diverse toolsets. 9. For practitioners: code and demo are made available, enabling exploration of this next-gen agentic architecture. 10. In short: if you’re building scalable, tool-driven AI agents, DeepAgent offers a powerful blueprint for stepping beyond rigid workflows into truly flexible, agentic reasoning.

31

#ByteDance has released #BAGEL, an open-source #multimodal foundation model designed for unified understanding and generation across text, image, and video. Unlike models that treat these modalities in isolation, BAGEL handles them within a shared transformer architecture, allowing for precise visual edits, photorealistic scene generation, and long-context reasoning across sequences. One of BAGEL’s core strengths is its ability to maintain #artistic and structural intent during image manipulation. Rather than treating edits as surface-level changes, it interprets context, such as #lighting, #composition, and narrative logic, to produce outcomes that feel deliberate and visually coherent. This supports workflows where #creative control and continuity matter, including those involving multi-step transformations or variations within the same conceptual frame. Beyond still imagery, BAGEL was trained on interleaved data that includes video and web-based visual instruction, giving it the ability to understand and predict #temporaldynamics. It handles complex #visuallogic across sequences, including future frame prediction and spatial transformations, which are essential for building #interactive, time-sensitive visual systems. This capability extends to what is often referred to as “#worldmodeling.” BAGEL can generate environments and objects that follow internal consistency across time, space, and user interaction, a requirement for real-time, immersive experiences. The model doesn’t just render appealing visuals; it understands how they evolve and relate within a given system, enabling environments that behave as well as they look. Evaluation results show that BAGEL performs well across a range of benchmarks, from classical image editing to tasks involving compositional reasoning and grounded knowledge. It achieves this through a bottleneck-free design that *avoids separating understanding and generation, allowing both to inform each other within the same context.* BAGEL is open-source and publicly available, offering researchers, developers, and production teams the ability to fine-tune or extend it within their own systems. BAGEL and the Future of Visual Reasoning in Creative Workflows (ScalaBle PerceptuAl Generative ModEL, as world foundational model) #Research Paper: https://lnkd.in/dWqVHBti Website: https://bagel-ai.org/

1

🧠 ML Interview Question from AIML.com: 𝐇𝐨𝐰 𝐝𝐨 𝐕𝐢𝐬𝐢𝐨𝐧 𝐓𝐫𝐚𝐧𝐬𝐟𝐨𝐫𝐦𝐞𝐫𝐬 (𝐕𝐢𝐓) 𝐰𝐨𝐫𝐤? 📖 Article Link: https://lnkd.in/gVcCY-8s For nearly a decade, CNNs were the de-facto models for computer vision. Then a paper from Google asked: "𝘞𝘩𝘢𝘵 𝘪𝘧 𝘸𝘦 𝘫𝘶𝘴𝘵 𝘵𝘳𝘦𝘢𝘵𝘦𝘥 𝘢𝘯 𝘪𝘮𝘢𝘨𝘦 𝘢𝘴 𝘢 𝘴𝘦𝘲𝘶𝘦𝘯𝘤𝘦 𝘰𝘧 𝘸𝘰𝘳𝘥𝘴?" By breaking images into patches and applying the Transformer architecture, ViT proved that global attention can match or exceed local convolutions when you have enough data. It sparked a paradigm shift that unified vision and language models. This article breaks down the architecture of Vision Transformers that changed the field of Computer Vision: 👉 Introduction of Vision Transformers 👉 How Vision Transformers Work 👉 Difference between ViT and CNNs 👉 Major ViT variants (Base, Large, Swin, BEiT) and their comparison 📽️ Videos explaining ViT in detail -- 🚀 Preparing for Machine Learning / LLM interviews? Join AIML.com ➡️ 400+ most asked Machine Learning Interview Questions ➡️ 600+ Quiz questions across AI / ML topics for hands on practice 🌐 AIML.com, the world's largest repository of Machine Learning Interview Questions - 𝐁𝐮𝐢𝐥𝐭 𝐛𝐲 𝐥𝐞𝐚𝐫𝐧𝐞𝐫𝐬, 𝐟𝐨𝐫 𝐥𝐞𝐚𝐫𝐧𝐞𝐫𝐬 #AIML_COM #VisionTransformer #ComputerVision #CNN #Interview #job

7

I Turned 80 Datasets into One Insight Engine 80 messy datasets. One insight engine. While working as a Research Assistant at HK PolyU, I had to process:    Survey scores    Written feedback    CSVs, Word files    And a lot of confusion Instead of analyzing them one by one, I: ✅ Built a central pipeline to clean, label, and theme feedback ✅ Used regression and clustering to group patterns ✅ Validated insights with 25% more accuracy The data wasn’t just organized — It became actionable intelligence. That’s the magic of good AI research: Not more models, just smarter preprocessing. Building something research-heavy and need structure? DM me. I build systems that see the signal. #DataScience #NLP #MachineLearning #AIResearch #AcademicAI #AIEngineer #Clustering #InsightToAction

2

In Retrieval-Augmented Generation (RAG), the LLM generates the answer — but the retriever decides what it sees. Term-based retrieval (like TF-IDF) prioritizes keyword frequency and speed. Embedding-based retrieval prioritizes semantic meaning. Hybrid search combines both to balance precision, recall, performance, and cost. If retrieval quality is weak, even the strongest model will produce weak answers. Better context → better reasoning → better responses. A technical breakdown of how retrieval algorithms shape LLM output: https://hubs.la/Q044DgDJ0 #RAG #LLMs #AIEngineering #VectorSearch #HybridSearch #MachineLearning #TribalScale

5

OpenAI Proposes National Strategy to Accelerate Korea’s AI Development ▶ OpenAI unveiled a policy report, “AI in Korea: OpenAI’s Economic Blueprint,” outlining its strategy for Korea’s AI growth. ▶ The plan calls for a “dual-track strategy” combining domestic AI ecosystem building with global cooperation. ▶ Focus areas include GPU and compute infrastructure, large-model deployment, and data governance frameworks. ▶ OpenAI cited Korea’s partnerships with Samsung, SK, and the Ministry of Science and ICT (MSIT) as key to its global “Stargate” AI infrastructure project. ▶ This marks the first country-level collaboration by OpenAI in the Asia-Pacific region. ▶ The report proposes phased cooperation, starting with SMEs, healthcare, education, and industry. ▶ Over time, the approach could evolve into a comprehensive “AI national package” integrating policy, technology, and finance. ▶ OpenAI said Korea’s strengths in semiconductors, talent, and digital infrastructure position it to become a global AI leader. ▶ “Korea can move from being a technology taker to a technology provider,” said Chris Lehane, OpenAI’s Chief Global Affairs Officer. 🔗 Link to article: https://lnkd.in/eqDEWCT2 📚 The AI Policy Newsletter: https://lnkd.in/eS8bHrvG

11

1 Comment

Looking for a quick overview of Agentic RL for LLMs? Sharing a recent blog post with (interactive) flashcard-style summaries of techniques from "The Landscape of Agentic Reinforcement Learning for LLMs: A Survey (Zhang et al., 2026)," covering planning, tool-use, memory, self-improvement, and reasoning. Link in comments!

14

1 Comment