Techniques are disclosed relating to machine learning techniques for performing user authentication based on the manner in which a user interacts with a client device, including the use of Siamese networks to detect unauthorized use of a device and/or account. In some embodiments, a server system may receive a request to authorize a transaction associated with a user account. The request may include transaction details and, separate from those transaction details, interaction data indicative of a manner in which a requesting user interacts with a client device during a user session. The server system may apply a machine learning model to the interaction data to create an encoding value that is based on the manner in when the requesting user interacts with the client device during the user session. The server system may then compare the encoding value to a reference encoding value and, based on the comparison, determine whether to authorize the transaction.