can i create a suitelet with pagination which display 100 results in firts page as a sublist and i have select checkbox for each line, when user goes to next page for results can i retain the selected values in first 100 results which user is selecting

I would recommend not to retain the user selection. Even NetSuite does not retain the user selection when the page changes.

You can do it but you'd need to hold selected values in a hidden field to ensure you do not lose anything and update that field whenever something is selected / deselected. Also when you move to a new list of rows populate the tick box based on those values held in your hidden field. I would typically avoid it and preface the SuiteLet with some kind of user selectable filter options to reduce your result set. It's not "easy" but to be fair when is anything customised in NS easy? 😉

You can hide/show lines with jQuery and handle the hiding/showing on field change (page number). All the data is still there, the jQuery just hides different groups.

@Sandii is there any sample code done using jQuery i would like to check that

//listens fieldChanged
    function handlePageChange(rec, context) {
        var currPage = Number(rec.getValue(context.fieldId));
        var multiplier =100;
        var minRow = (currPage - 1) * multiplier;
        var maxRow = (currPage * multiplier) - 1;
        var lineCount = rec.getLineCount('custpage_sublistid');
        for (var i = 0; i < lineCount; i++) {
            (i >= minRow && i <= maxRow)
                ? updateRowDisplay(i, '')
                : updateRowDisplay(i, 'none');
        }
    }

    //listens handlePageChange
    function updateRowDisplay(line, displayType) {
        jQuery('#custpage_sublistidrow' + line).css('display', displayType);
    }

something like that on your client, obviously need more in the fieldChanged, you also need to handle some pagination on pageInit to hide lines after the first 100 (or whatever 'multiplier' you use)