Renzo Demartini Talk to me

Insights / Governance and privacy

AI and customer data: what a board should ask

Before customer information goes into an AI tool, a board should know what each task sends, which tool receives it and who answers for it. Six questions, one decision record.

Exhibit 1What may this customer enquiry send to an AI tool?

The AI tool sees only what the decision record allows, and bank details never reach it.

Illustrative figures, not client data.

In short

  • A board doesn’t approve every AI task, but it should know which tasks send customer information, to which tools and who answers for each.
  • Send only what the task needs, and check the exact tool and account: a consumer chatbot is not a business tool with a data agreement.
  • Ask for a short decision record before live use, with a named owner, and a reporting route for when something goes wrong.

Most AI use in a business starts small: someone tries a tool on a real task. That is how customer information ends up in places nobody approved.

For a board, the question is whether management can show what each task sends, where it goes and who is accountable. A board can’t check every workflow, but it can ask for the record behind each one.

Six questions for management
Exhibit 2

A board needs six answers, and one decision record holds them

Ask managementA good answer shows
Which tasks send customer or staff information to an AI tool?Who approved each one
What is the minimum information each task needs?What is masked or kept out
Which exact product and account receives it?Whether its terms allow training on our data, overseas processing or long retention
Who checks the output?They check it before it reaches a customer or a record
What happens when someone uploads the wrong thing?When the board was last told about one
Who keeps the decision record current?When it was last reviewed

These are the questions I would put to management before any customer data reaches an AI tool. Management should be able to answer each one from a short decision record. If it can’t, that is the finding.

More detail

Directors oversee how the business manages its risks, and privacy is one of them.

Start with what the task needs
Exhibit 3

A good test result is not permission to go live

Test with sample enquiries

  • Sample enquiries that look like your real ones
  • Include incomplete and confusing ones
  • No real names or account numbers

Live use is a separate decision

  • A good result with made-up data isn’t permission
  • Don’t connect the mailbox until the record is written
Exhibit 4 · Try itSort what a task would send to AIPick the task and the kind of tool, then tick the information it would send. Each item lands in a lane with the reason, and a decision record builds below for you to copy.Open the sorter ↓Close ↑

General guidance for planning, not legal advice. Whether the Privacy Act applies to you, and whether a particular use is permitted, depends on your business and the task. Check the OAIC guidance or get privacy advice. Examples are sample data.

Look at three things before customer data goes into an AI tool: the task, the information it needs, and the controls and terms of that product.

More detail

Take an assistant that sorts account enquiries into billing, service and general questions. To test the categories, you don’t need real names, contact details or account numbers. Write sample enquiries that look like your real ones, including incomplete and confusing ones.

When a real task lands, staff need to know what to do. A workable instruction names the approved tool and account, the task, the information allowed, the reviewer and who to tell if something goes wrong. For the enquiry test it might read:

‘Use the approved test workspace with sample enquiries only. Ask for a category and a reason. Don’t connect email or upload attachments. Log errors in the pilot sheet for the operations owner.’

It’s a narrow test instruction, not an AI policy, but people can follow it.

What Australian privacy law expects
Exhibit 5

Three privacy principles matter most when personal information goes into an AI tool

  • 01

    Use and disclosure (APP 6)

    Generally, use information for the purpose you collected it for, or a related purpose the customer would reasonably expect.

  • 02

    Overseas disclosure (APP 8)

    If the provider handles prompts or files outside Australia, you need to account for that.

  • 03

    Security (APP 11)

    Take reasonable steps to protect the information, including who can reach the tool and what it keeps.

Source: OAIC: guidance on privacy and the use of commercially available AI products

The Privacy Act 1988 and its 13 Australian Privacy Principles (APPs) apply to most businesses with annual turnover above A$3M, and to some smaller ones. If you’re covered, putting personal information into an AI tool has to fit the APPs.

The OAIC’s guidance on commercially available AI products recommends, as best practice, not entering personal information, particularly sensitive information, into publicly available AI tools.

More detail

Some smaller businesses are covered whatever their turnover, such as health service providers and businesses that trade in personal information. Health information is sensitive information, with stricter rules. Tax file numbers carry their own limits under the TFN Rule.

This is practical guidance, not legal advice. Whether you’re covered and whether a use is permitted depends on your circumstances, so get privacy advice where it’s unclear. A business outside the Act still holds its customers’ trust.

Check the tool, not the brand
Exhibit 6

The exact product and account decide the risk, not the brand

  • 01

    Training

    Whether prompts and files train the model, and whether that can be switched off.

  • 02

    Retention

    How long prompts, attachments, logs and backups are kept, and where.

  • 03

    Who can see them

    Who at your business and at the provider can see them.

  • 04

    Connected tools

    Whether connected tools pass the information anywhere else.

Hosting location is one part of this, not the answer: a local server doesn’t help if the wrong person can open customer records.

More detail

De-identification needs care too. Removing a name can still leave enough detail to identify a person or a business in a small market.

Write a decision record before going live
Exhibit 7

One page of six answers, kept current by one named person

  1. 01The task

    The one job the workflow does.

  2. 02The minimum information it needs

    Only what the task requires, and nothing more.

  3. 03The approved tool and account

    The exact product and account, not the brand.

  4. 04Who can see the output

    The people allowed to see what it produces.

  5. 05Who checks it

    A person checks it before it reaches a customer or a record.

  6. 06What happens when it fails

    The reporting route, and who acts.

Name the person who keeps these answers current, and revisit the record whenever the task changes.

Exhibit 8 · The workflow

An enquiry sorted without the personal details

What gets caught: Bank details pasted into an enquiry is caught at step 2, ‘Strip what the task doesn’t need’.

  1. Claude reads
  2. Rules check
  3. A person decides

You decided what the AI tool may see.

Contact details masked first. Bank, tax and health details never.

Then an enquiry arrives.

A credit for a missed service visit. It looks routine.

Decision record

What the AI tool may seeAccount enquiries

Kept current by a named manager

  • Contact detailsMasked first
  • Bank detailsNever
  • Tax or health detailsNever
  • ToolApproved business account

Account enquiry

Website enquiryNo attachment

Credit for a missed service visit

  • AccountCommercial, three sites
  • ContactName, mobile 04•• ••• 218, email, company
  • NoteRefund the credit to BSB 06•-••• · •••• 1180

Plain rules check it first.

Before Claude (Anthropic’s AI model) sees a word, the details the task doesn’t need come out.

Decision recordAllowedDecision record
ArrivedAccount enquiry
Contact detailsAllowed: Masked first
Arrived: 4 fields, now masked. Matches
Bank detailsAllowed: Never
Arrived: BSB 06•-••• · •••• 1180. Doesn’t match
Tax or health detailsAllowed: Never
Arrived: None. Matches
  • 4 contact fields masked
  • Bank details found
  • No tax or health details

The risk, in details.

The customer typed bank details into the form.

What the AI tool would have seen

Kinds of detail never allowed1a BSB and account number

Contact fields masked
4
Bank details in the note
BSB and account number
Sent to the AI tool
Nothing

The numbers are masked and the enquiry goes to a person instead.

It waits for the coordinator.

Nothing goes to the AI tool until a person decides. Here, that’s you.

Accounts coordinatorMessage from the decision-record check

Held. Nothing sent to the AI tool.

Needs your decision

This enquiry had bank details in it. They’re masked now. Reply from the CRM yourself, or send the masked text to Claude to sort?

You decide: pick one to see what happens, or the run goes with “Reply from the CRM yourself”.

Held from the AI tool.

Rules check first. A person decides. Claude (Anthropic’s AI model) sees only what’s allowed.

With the coordinatorNothing from this enquiry reaches the AI tool.

The coordinator’s own reply from the CRM. Nothing reaches the AI tool.

CRM emailSentSample

From
Accounts coordinator
To
Customer, commercial account
Subject
Re: Credit for a missed service visit

Hi, thanks for letting us know, and sorry we missed the visit. We’ll credit it to your account and you’ll see it on your next statement.

For your security, please don’t send bank details by email. We’ve removed them from your message.

Written and sent by the accounts coordinator

  • Plain rules masked 4 contact fields, found a BSB and account number in the note, and held the enquiry.
  • Claude isn’t reached: nothing goes to it until the details are masked and a person says so.
  • The coordinator checked the enquiry and replied from the CRM, where the contact details stay.

With four contact fields masked and no bank details, Claude sorts it as a billing query with its reason, and the coordinator checks it and replies from the CRM.

Sorted as billingClaude sees the request, the account type and the note, with every detail masked.

Claude’s category on the CRM record, from the masked text only.

CRMSample

Account enquiryBilling query

Category
Billing query, set by Claude
Reason
A credit asked for a missed service visit
Claude saw
The request, the account type and the note, masked

Masked text: “Credit for a missed service visit. Commercial, three sites. [contact masked]. Refund the credit to [bank details masked].”

Sorted by ClaudeChecked and replied by the coordinator

  • Plain rules masked 4 contact fields, found a BSB and account number in the note, and held the enquiry.
  • Claude sorted the masked text as a billing query, with its reason: a credit asked for a missed service visit.
  • The coordinator checked Claude’s category and replied from the CRM.

With four contact fields masked and no bank details, Claude sorts it as a billing query with its reason, and the coordinator checks it and replies from the CRM.

Sample data: a fictional customer and enquiry.

ClaudeWhere Claude helps

  • Sorts or summarises text after identifying details are removed
  • Gives a reason with each category so a person can check it
  • Flags text that looks like bank, tax or health details for a person

RulesWhat the rules check

  • Only fields named in the decision record leave your systems
  • Tax file numbers, bank details and health details are blocked or masked
  • Only the approved tool and account can be used for the task

PersonWhat a person decides

  • A named manager decides what the task may send and keeps the record current
  • A coordinator checks every output before anything reaches a customer
  • A named person handles a wrong upload and decides what happens next

NoWhat Claude doesn’t do

  • Decide whether the Privacy Act applies to your business
  • Send a reply to a customer on its own
  • Change a customer’s bank details, prices or records

Sample data. Claude reads, plain rules check, and you make the decision. Press run to watch it get caught.

Often the answer is to split the work. An enquiry assistant may only need the request and the type of account to suggest routing, while the contact details stay in your CRM for the coordinator.

In my workflows, a person approves anything that sends.

More detail

Revisit the record whenever the task changes: letting the same assistant read attachments or draft replies is a new scope.

What to do first
Exhibit 9

Five steps before the first live task, none needing customer records

  1. 01Map one task’s information flow

    What it would send, and to which tool, drawn without customer records.

  2. 02Test with sample enquiries

    Made-up enquiries that look like your real ones, in an approved test workspace.

  3. 03Write the decision record

    The six answers, and the named person who keeps them current.

  4. 04Set the reporting route

    Before the pilot: record the tool, the time and the type of information, without forwarding the material.

  5. 05Know what happens next

    The responsible person looks at containment, deletion options and the Notifiable Data Breaches scheme. Deleting the chat may not remove every copy.

You can map one task’s information flow without sending any customer records. For who runs a workflow once it’s live, see what a system costs to own after the build.

Common questions

Can staff paste customer details into a free AI chatbot?

It’s best not to. The OAIC recommends, as best practice, not entering personal information into publicly available AI tools, and a free or personal account may keep prompts and use them for training. Test with sample data instead, and use a business account with a data agreement, or an AI step inside your own systems, for live work.

Does the Privacy Act apply to our business?

Generally it applies to businesses with annual turnover above A$3M, and to some smaller ones regardless of turnover, including health service providers and businesses that trade in personal information. The OAIC website explains the tests. If you’re unsure, get advice. Either way, customers expect their details to be handled with care.

What if someone puts customer data into the wrong AI tool?

Report it straight away through the route you set up before the pilot. Record the tool, the time and the type of information, without spreading it further. The responsible person assesses containment, deletion options and whether the Notifiable Data Breaches scheme applies. Don’t assume deleting the conversation removes every copy the provider holds.

Renzo Demartini, Chartered Accountant and finance executive, Canberra

Renzo Demartini

Chartered Accountant (CA ANZ) and finance executive, based in Canberra. Practical AI and automation for SMEs, and adviser to owners, CEOs and boards on integration and finance.

Need one workflow reviewed at a fixed price? Start with a Works Better workflow audit. Changing how the whole finance function runs? Start a conversation.

What’s happening in your business?

Tell me in a few sentences. I reply personally within one business day.