would it be appropriate to compare this to passing a copy of a pointer to an object
In my opinion, yes. Nonprimitive variables (and other expressions) point to things. You can give a method a copy of their value, and that method can then change the state of the thing pointed to, but *not* change the value of the caller's variable (because it only has a copy of that variable's value). Of course there is no pointer arithmetic...
Wouldn't that basically mean that regular variables are passed by value, and object are passed by reference
in essence?
No - but the reason is a bit subtle. Variables are not passed at all: their values are. These values are known in Java by a variety of terms - references, reference values, pointers. That only values are passed around (and not variables, operations on which would be visible in the caller's context) keeps the semantics simple. There is little (any?) difference between passing, returning and assigning reference values (pointers) and doing the same things with their primitive brethren.
passing an object By Value and then using its member functions would have absolutely no result outside of the method because you were only altering the state of a copy
Yes - and that's a perfectly good way of doing things in other languages. But, to repeat, in Java objects *aren't* passed: reference values (pointers) are.
You had to either pass By Reference or pass a pointer (the same thing usually)
Java is a nice illustration of how the semantics differ. If a method gets a hold of a reference to one of the caller's variables, it can alter the value of that variable. If it only gets the value of a pointer, it can alter the state of the thing pointed to but not the value of variables in other contexts. Java doesn't have the former, but does have the latter.
-----
I get the impression that you aren't overly worried by this, but just want to clarify how the method calling semantics compares with others you have used. For my money, think "pointers" and you won't go far wrong.
You should also bear in mind that not only do method calling semantics vary from language to language, but so does the terminology used to describe them! When this (passing a copy of a pointer) technique was first elaborated (in the early 1970's?) its inventors were so struck by the fact that you could alter the state of objects without having to pass that entire state to a method that they chose to call it "pass by reference" and this usage has stuck in some communities. But it does obscure a real difference between PBR and passing (a copy of the value of) a pointer: they might be available as options to allow a programmer to achieve "the same thing", but they have very different effects - that the programmer has to be aware of - on the value of variables in the caller's context. Java has chosen not to implement PBR as a way of "keeping things simple".