AI-103 Practice Question 94: Computer Use Tool Guardrails
Microsoft Azure AI Apps and Agents Developer Associate (AI-103) · Test 10 · question 4 of 10
An operations team wants an agent to submit maintenance tickets in a decades-old internal web portal. The portal exposes no API of any kind, only a browser-rendered UI, so no OpenAPI, function-calling, or MCP tool has anything to call. The team wants the agent to work the portal the way a person would: look at the screen and simulate clicks and keystrokes.
Which tool fits, and what precaution is essential before it runs unattended?
Show answer and explanation
Answer:
The computer use tool exists for interfaces with no API: it lets an agent "interpret screenshots and automate UI interactions like clicking, typing, and scrolling" through a screenshot-action-screenshot loop. Microsoft's guidance is explicit that it should run only "on low-privilege virtual machines that do not contain sensitive data or credentials," and that the API raises pending_safety_checks for things like prompt injection or sensitive-domain access; the calling application must "pause automation, require an end user to review the request, and only continue after you send acknowledged_safety_checks."
- A is wrong because the code interpreter's managed sandbox is built for running and analyzing code and files, not for vision-driven UI control, and it isn't where Microsoft's UI-automation guardrails apply. - B is wrong on two fronts: a function tool needs something to call, and this portal has no API (a scripted UI walkthrough isn't what the tool invokes), and embedding a standing, high-privilege service credential in a hosted function directly contradicts the no-sensitive-access guidance for automated UI control. - D is wrong because the OpenAPI tool consumes a real OpenAPI specification for an existing API; hand-writing a specification over rendered HTML pages describes no such API and isn't a supported or reliable substitute.
Reference: Use the computer use tool for agents