Skip to content

inportb/webquery

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 
 
 

Repository files navigation

WebQuery

WebQuery is a Pythonic way to interact with API's using SQL. In other words, Python + SQLite ≈ YQL.

Dependencies:
  • yaml
  • lxml
  • apsw

Interactive shell:

./webquery.py *.yml

Using a pipe:

echo "SELECT * FROM delicious_feeds_popular WHERE query='test' ORDER BY url DESC; SELECT url,title FROM microsoft_bing_web WHERE query='test' LIMIT 5;" | ./webquery.py *.yml

Limiting requests (useful for sorting):

echo "SELECT webquery('microsoft_bing_web','truncate',50); SELECT title,url FROM microsoft_bing_web WHERE query='sushi' ORDER BY title;" | ./webquery.py *.yml

In Python:

import apsw, webquery
db = apsw.Connection(':memory:')
webquery.attach(db)
cur = db.cursor()
cur.execute('CREATE VIRTUAL TABLE microsoft_bing_web USING webquery(microsoft.bing.web.yml);')
cur.execute('SELECT webquery(?,?,?);',('microsoft_bing_web','truncate',50))
for title,url in cur.execute('SELECT title,url FROM microsoft_bing_web WHERE query=? ORDER BY title;',('sushi',)):
    print title,'>',url

In SQL:

-- load definition from a file
CREATE VIRTUAL TABLE table1 USING webquery(microsoft.bing.web.yml);
-- load definition from the Web
CREATE VIRTUAL TABLE table2 USING webquery(http://www.example.com/microsoft.bing.web.yml);
-- get configuration variable
SELECT webquery('table1','truncate');
-- set configuration variable and return previous value
SELECT webquery('table1','truncate',50);
nb:
  • The ORDER BY clause does not map to server-side sorting; it sorts the results that are returned by the server.
  • Request truncation (by the truncate variable) always takes precedence over the LIMIT clause, preventing unbounded requests while performing a sorted query. It does not guarantee that the result set be smaller than the set value, so the LIMIT clause is still useful.

Releases

No releases published

Packages

No packages published

Languages