When to use references vs. pointers

I comprehend the syntax and general semantics of pointers versus references, however how might I choose when it is pretty much suitable to utilize references or pointers in an API?

Normally a few circumstances need either (operator++ needs a reference contention), yet overall I'm discovering I like to utilize pointers (and const pointers) as the syntax is certain that the variables are being passed destructively.

For example in the accompanying code:

void add_one(int& n) { n += 1; }

void add_one(int const n) { n += 1; }

int main() {

  int a = 0;

  add_one(a); // Not clear that a may be modified

  add_one(&a); // 'a' is clearly being passed destructively

}

With the pointer, it's consistently (more) clear what's happening, so for APIs and such where clarity is a major concern are pointers not more proper than references? Does that mean references ought to possibly be utilized when essential (for example operator++)? Are there any exhibition worries with either?