A Hyperelliptic Smoothness Test. I

H. W. Lenstra , J. Pila , Carl Pomerance

Abstract

This series of papers is concerned with a probabilistic algorithm for finding small prime factors of an integer. While the algorithm is not practical, it yields an improvement over previous complexity results. The algorithm uses the jacobian varieties of curves of genus 2 in the same way that the elliptic curve method uses elliptic curves. In this first paper in the series a new density theorem is presented for smooth numbers in short intervals. It is a key ingredient of the analysis of the algorithm.