Post reply

Loading Web pages directly from SQL Server

  • March 24, 2010 at 2:52 am

    #220309

    I have an application that takes a URL and store it's contents for future Full-Text Search operations.

    I prefer to implement the contents retrieval directly from SQL Server.

    Can you advise me of the best practice for acheiving this, whether to load source code as text or to store the HTML file as binary data.

    Your advise please

    Thanks,

  • March 24, 2010 at 6:53 am

    #1138864

    Nawar.Tabaa (3/24/2010)

    I prefer to implement the contents retrieval directly from SQL Server. Can you advise me of the best practice for acheiving this, whether to load source code as text or to store the HTML file as binary data.

    Are you talking about retrieving the data from websites using SQL Server or just using SQL Server for storage?

    How large do you expect each data item to be on average? If less than about 256KB, it might be best stored as NVARCHAR(MAX) or VARBINARY(MAX). If larger than 1MB, consider FILESTREAM.

    See FILESTREAM White Paper

    Paul White
    SQLPerformance.com
    SQLkiwi blog
    @SQL_Kiwi

  • March 24, 2010 at 7:08 am

    #1138887

    Thanks Paul, apart from the storage is there any way to load HTML documents from the web like those exists for documents on local file system / network share?

    SELECT * FROM OPENROWSET(BULK N'F:\test.htm', SINGLE_BLOB)

    Thanks,

  • March 24, 2010 at 7:23 am

    #1138904

    Nawar.Tabaa (3/24/2010)

    Thanks Paul, apart from the storage is there any way to load HTML documents from the web like those exists for documents on local file system / network share?

    SELECT * FROM OPENROWSET(BULK N'F:\test.htm', SINGLE_BLOB)

    Thanks,

    Yes, but my view is that this task has nothing to do with SQL Server and is better done in an external application.

    Paul White
    SQLPerformance.com
    SQLkiwi blog
    @SQL_Kiwi

Viewing 4 posts - 1 through 3 (of 3 total)

You must be logged in to reply to this topic. Login to reply