<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0"
	xmlns:content="http://purl.org/rss/1.0/modules/content/"
	xmlns:wfw="http://wellformedweb.org/CommentAPI/"
	xmlns:dc="http://purl.org/dc/elements/1.1/"
	xmlns:atom="http://www.w3.org/2005/Atom"
	xmlns:sy="http://purl.org/rss/1.0/modules/syndication/"
	xmlns:slash="http://purl.org/rss/1.0/modules/slash/"
	>

<channel>
	<title>StephenChan&#039;s Tech Space &#187; Python</title>
	<atom:link href="http://blog.endlesscode.com/category/python/feed/" rel="self" type="application/rss+xml" />
	<link>http://blog.endlesscode.com</link>
	<description>Stay Hungry. Stay Foolish.</description>
	<lastBuildDate>Tue, 25 Oct 2011 01:15:07 +0000</lastBuildDate>
	<language>en</language>
	<sy:updatePeriod>hourly</sy:updatePeriod>
	<sy:updateFrequency>1</sy:updateFrequency>
	<generator>http://wordpress.org/?v=3.0.1</generator>
		<item>
		<title>基于coroutine的gevent</title>
		<link>http://blog.endlesscode.com/2011/10/22/gevent-lib/</link>
		<comments>http://blog.endlesscode.com/2011/10/22/gevent-lib/#comments</comments>
		<pubDate>Sat, 22 Oct 2011 13:13:51 +0000</pubDate>
		<dc:creator>Stephen</dc:creator>
				<category><![CDATA[Python]]></category>

		<guid isPermaLink="false">http://blog.endlesscode.com/?p=1209</guid>
		<description><![CDATA[coroutine也是一种并发模型，但不同于thread和callback，它的所有task都是可以在一个线程里面执行，然后可以通过在一个task里面主动放弃执行来切换到另一个task执行，它的调度是程序级的，不像thread是系统级的调度。gevent就是一个基于coroutine的python网络开发框架，不像twisted那样集成了很多库和协议，gevent非常精简，当然文档也很少，在性能上的话，看了PyCon上的视频，对比了twisted和其他的几个库，在内存的性能上非常地优秀，代码简洁而且也支持多核。 做了个相当简单的ChatService，有登录、群发消息、退出这3个消息，协议就是用简单的json了，有时间再试试xmpp看看。&#8221;monkey.patch_all()&#8221; 这个调用就把python原生的一些socket对象和方法替换成非阻塞的异步调用，写起来感觉还是很方便。 #!/usr/bin/env python #-*- coding:utf-8 -*- import sys import simplejson from gevent import monkey; monkey.patch_all(); from gevent.server import StreamServer from common import Protocol, gen_pro_data, parse_pro_data class ChatService: def __init__(self): self._clients = {} self._handlers = { Protocol.Connect : self.connect, &#8230; <a href="http://blog.endlesscode.com/2011/10/22/gevent-lib/">Continue reading <span class="meta-nav">&#8594;</span></a>]]></description>
			<content:encoded><![CDATA[<p>coroutine也是一种并发模型，但不同于thread和callback，它的所有task都是可以在一个线程里面执行，然后可以通过在一个task里面主动放弃执行来切换到另一个task执行，它的调度是程序级的，不像thread是系统级的调度。gevent就是一个基于coroutine的python网络开发框架，不像twisted那样集成了很多库和协议，gevent非常精简，当然文档也很少，在性能上的话，看了PyCon上的视频，对比了twisted和其他的几个库，在内存的性能上非常地优秀，代码简洁而且也支持多核。</p>
<p>做了个相当简单的ChatService，有登录、群发消息、退出这3个消息，协议就是用简单的json了，有时间再试试xmpp看看。&#8221;monkey.patch_all()&#8221; 这个调用就把python原生的一些socket对象和方法替换成非阻塞的异步调用，写起来感觉还是很方便。</p>
<p><span id="more-1209"></span></p>
<pre class="brush:python">#!/usr/bin/env python
#-*- coding:utf-8 -*-

import sys
import simplejson
from gevent import monkey; monkey.patch_all();
from gevent.server import StreamServer
from common import Protocol, gen_pro_data, parse_pro_data

class ChatService:
    def __init__(self):
        self._clients = {}
        self._handlers = {
            Protocol.Connect : self.connect,
            Protocol.SendMsg : self.sendmsg,
            Protocol.Quit : self.quit,
        }

    def connect(self, sock, data):
        userinfo = self._clients.get(sock, None)
        if userinfo: return
        self._clients[sock] = data['name']
        print "New User : %s" % (data['name'],)

    def sendmsg(self, sock, data):
        msg = gen_pro_data(data)
        for s, u in self._clients.items():
            if sock != s:
                s.send(msg)

    def quit(self, sock, data=None):
        if sock in self._clients:
            print "User [%s] quit." % (self._clients[sock],)
            del self._clients[sock]
            sock.close()

    def process(self, sock, promsg):
        cmd = promsg['cmd']
        func = self._handlers[cmd]
        func(sock, promsg)

# server chat service
chatservice = ChatService()

def serve(sock, addr):
    while True:
        try:
            length = ord(sock.recv(1))
            content = sock.recv(length)
            # protocol :  length + json({'cmd' : 'sendmsg', 'msg' : 'hallo'})
            promsg = parse_pro_data(content)
            chatservice.process(sock, promsg)
        except:
            chatservice.quit(sock)
            break

if __name__ == "__main__":
    host = sys.argv[1]
    port = int(sys.argv[2])
    print "Start chat server on %s:%s" % (host, port)
    try:
        chatserver = StreamServer((host, port), serve)
        chatserver.serve_forever()
    except:
        chatserver.kill()</pre>
<p>写了一个简单的测试，创建了5000个连接，然后定时发消息，服务器也没显得有什么压力。</p>
<pre class="brush:python">def test():
    import gevent
    from gevent.pool import Pool

    def robot(idx, host, port):
        # client chat service
        chatservice = ChatService(host, port)
        chatservice.connect("user%s" % (idx,))
        while True:
            chatservice.sendmsg("hallo from %s" % (idx,))
            gevent.sleep(3)
    host = sys.argv[1]
    port = int(sys.argv[2])
    count = 5000
    pool = Pool(5000)
    for i in range(count):
        pool.spawn(robot, i, host, port)
    pool.join()</pre>
]]></content:encoded>
			<wfw:commentRss>http://blog.endlesscode.com/2011/10/22/gevent-lib/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>URL映射</title>
		<link>http://blog.endlesscode.com/2010/06/04/url-dispatch/</link>
		<comments>http://blog.endlesscode.com/2010/06/04/url-dispatch/#comments</comments>
		<pubDate>Fri, 04 Jun 2010 08:23:34 +0000</pubDate>
		<dc:creator>Stephen</dc:creator>
				<category><![CDATA[Python]]></category>

		<guid isPermaLink="false">http://blog.endlesscode.com/?p=782</guid>
		<description><![CDATA[url映射是目前流行的框架都提供的基本功能，参考了django的url映射的源码，用python实现url映射的主要逻辑还是比较简单的。主要是由两个部分组成： 将处理函数的命名空间解析成模块，使用__import__，导入模块，获取处理函数的引用（见get_callable()）； 匹配访问路径，提取参数，使用正则表达式进行匹配。 # add url_base as namespace to handlers in url_patterns def include(url_base, url_patterns): for index, src_pattern in enumerate(url_patterns): pattern = list(src_pattern) pattern[1] = "%s.%s" % (url_base, pattern[1]) url_patterns[index] = tuple(pattern) return url_patterns # get module name and func &#8230; <a href="http://blog.endlesscode.com/2010/06/04/url-dispatch/">Continue reading <span class="meta-nav">&#8594;</span></a>]]></description>
			<content:encoded><![CDATA[<p>url映射是目前流行的框架都提供的基本功能，参考了django的url映射的源码，用python实现url映射的主要逻辑还是比较简单的。主要是由两个部分组成：</p>
<ul>
<li>将处理函数的命名空间解析成模块，使用__import__，导入模块，获取处理函数的引用（见get_callable()）；</li>
<li>匹配访问路径，提取参数，使用正则表达式进行匹配。</li>
</ul>
<pre class="brush:python"># add url_base as namespace to handlers in url_patterns
def include(url_base, url_patterns):
    for index, src_pattern in enumerate(url_patterns):
        pattern = list(src_pattern)
        pattern[1] = "%s.%s" % (url_base, pattern[1])
        url_patterns[index] = tuple(pattern)
    return url_patterns

# get module name and func name
def get_mod_func(handler):
    try:
        dot = handler.rindex(".")
    except ValueError:
        return handler, ''
    return handler[:dot], handler[dot+1:]<span id="more-782"></span>

# analyze handler path, return handler ref
def get_callable(handler):
    if not callable(handler):
        mod_name, func_name = get_mod_func(handler)
        mod = __import__(mod_name, globals(), locals(), [func_name])
        if func_name != '':
            func = getattr(mod, func_name)
            if not callable(func):
                raise AttributeError('%s.%s is not callable.' % (mod_name, func_name))
            return func
    return handler

# resolve path info, return handler and args
# path : path_info
# url_config : url_patterns, ref test()
# default_kwargs : default args passed to handler
def resolve(path, url_config, default_kwargs={}):
    import re
    for config in url_config:
        pattern, handler_name = config[0], config[1]
        other_args = config[2:]
        handler = get_callable(handler_name)
        match = re.compile(pattern).search(path)
        if match:
            kwargs = match.groupdict()
            # If there are any named groups, use those as kwargs, ignoring
            # non-named groups. Otherwise, pass all non-named arguments as
            # positional arguments.
            if kwargs:
                args = ()
                for arg in other_args: kwargs.update(arg)
            else:
                args = match.groups() + other_args
        kwargs.update(default_kwargs)
        return handler, args, kwargs
    else:
        raise 

def test():
    url_config = [
        (r"^/(?P&lt;username&gt;\w+)/blog/(?P&lt;blogid&gt;\d+)/?$", "user.test.blogview", {'test':True, 'arg0':'hello world'}),
        (r"^/user/(\d+)/?$", "lib.test2.userview", 'fuckgfw', 'fuckgov')
    ]
    url_config = include("app", url_config)
    handler, args, kwargs = resolve("/stephenchan/blog/890604/", url_config)
    handler(*args, **kwargs)

    handler, args, kwargs = resolve("/user/406098", url_config)
    handler(*args, **kwargs)

if __name__ == '__main__':
    test()</pre>
<p>测试的目录结构如下：</p>
<pre class="brush:plain">|-- app
|   |-- __init__.py
|   |-- __init__.pyc
|   |-- lib
|   |   |-- __init__.py
|   |   |-- __init__.pyc
|   |   |-- test2.py
|   |   `-- test2.pyc
|   `-- user
|       |-- __init__.py
|       |-- __init__.pyc
|       |-- test.py
|       `-- test.pyc</pre>
<p>两个测试的函数逻辑很简单：</p>
<pre class="brush:python"># app.lib.test2.userview
def userview(uid, arg1, arg2):
    print """
    userid = %s
    arg1 = %s
    arg2 = %s
    """%(uid, arg1, arg2)

# app.user.test.blogview
def blogview(username, blogid, test=False, arg0=""):
    print """
    username = %s
    blogid = %s
    test = %s
    arg0 = %s
    """ % (username, blogid, test, arg0)</pre>
<p>由于在匹配的结果中python正则表达式对象在matchObject.groups()返回的结果中包括了matchObject.groupdict()的结果，因此在匹配一个路径的时候难以将具名和非具名的参数分离，即具名参数和非具名参数只能以互斥的关系存在。test()函数中可以看到具名和非具名的两种路径的匹配设置。</p>
]]></content:encoded>
			<wfw:commentRss>http://blog.endlesscode.com/2010/06/04/url-dispatch/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
		<item>
		<title>Python的logging模块</title>
		<link>http://blog.endlesscode.com/2010/06/03/python-logging-module/</link>
		<comments>http://blog.endlesscode.com/2010/06/03/python-logging-module/#comments</comments>
		<pubDate>Thu, 03 Jun 2010 11:39:09 +0000</pubDate>
		<dc:creator>Stephen</dc:creator>
				<category><![CDATA[Python]]></category>

		<guid isPermaLink="false">http://blog.endlesscode.com/?p=763</guid>
		<description><![CDATA[初次使用logging模块觉得有点诡异，涉及到Logger、Handler、Level等概念，看代码最实际了： import logging import sys logger = logging.getLogger("endlesscode") formatter = logging.Formatter('%(name)-12s %(asctime)s %(levelname)-8s %(message)s', '%a, %d %b %Y %H:%M:%S',) file_handler = logging.FileHandler("test.log") file_handler.setFormatter(formatter) stream_handler = logging.StreamHandler(sys.stderr) logger.addHandler(file_handler) logger.addHandler(stream_handler) #logger.setLevel(logging.ERROR) logger.error("fuckgfw") logger.removeHandler(stream_handler) logger.error("fuckgov") 文档上已经说明得很清楚了，有几个比较特别的地方： 如果设置了setLevel的日志等级，则调用比设定低的日志记录函数则不会有任何的输出，比如: logger.setLevel(logging.ERROR) #没有生效，默认的info&#60;error logger.info("fuckgfw") 一个logger如果添加了多个handler(logger.addHandler)，记录的时候会同时在多个hanlder上输出日志。 日志的输出格式是通过logging.Formatter来设定的，其初始化的参数如下： logging有一个basicConfig的成员函数，主要是用来配置一些基本的信息： &#8230; <a href="http://blog.endlesscode.com/2010/06/03/python-logging-module/">Continue reading <span class="meta-nav">&#8594;</span></a>]]></description>
			<content:encoded><![CDATA[<p>初次使用logging模块觉得有点诡异，涉及到Logger、Handler、Level等概念，看代码最实际了：</p>
<pre class="brush:python">import logging
import sys 

logger = logging.getLogger("endlesscode")
formatter = logging.Formatter('%(name)-12s %(asctime)s %(levelname)-8s %(message)s', '%a, %d %b %Y %H:%M:%S',)
file_handler = logging.FileHandler("test.log")
file_handler.setFormatter(formatter)
stream_handler = logging.StreamHandler(sys.stderr)
logger.addHandler(file_handler)
logger.addHandler(stream_handler)
#logger.setLevel(logging.ERROR)

logger.error("fuckgfw")

logger.removeHandler(stream_handler)
logger.error("fuckgov")</pre>
<p>文档上已经说明得很清楚了，有几个比较特别的地方：<span id="more-763"></span></p>
<ul>
<li> 如果设置了setLevel的日志等级，则调用比设定低的日志记录函数则不会有任何的输出，比如:
<pre class="brush:python">logger.setLevel(logging.ERROR)
#没有生效，默认的info&lt;error
logger.info("fuckgfw")</pre>
</li>
<li> 一个logger如果添加了多个handler(logger.addHandler)，记录的时候会同时在多个hanlder上输出日志。</li>
<li>日志的输出格式是通过logging.Formatter来设定的，其初始化的参数如下：<br />
<img class="aligncenter size-full wp-image-770" title="formatter" src="http://blog.endlesscode.com/wp-content/uploads/2010/06/formatter.jpg" alt="formatter" width="747" height="431" /></li>
<li>logging有一个basicConfig的成员函数，主要是用来配置一些基本的信息：
<pre class="brush:python">logging.basicConfig(
    level=logging.ERROR,
    format='%(name)-12s %(asctime)s %(levelname)-8s %(message)s',
    datefmt='%a, %d %b %Y %H:%M:%S',
    filename=log_file,
    filemode='a')</pre>
<p>文档上的说明：</p>
<pre class="brush:plain">Does basic configuration for the logging system by creating a StreamHandler with a default Formatter and adding it to the root logger. The functions debug(), info(), warning(), error() and critical() will call basicConfig() automatically if no handlers are defined for the root logger.</pre>
<p>参数如下：<br />
<img class="aligncenter size-full wp-image-769" title="basiconfig" src="http://blog.endlesscode.com/wp-content/uploads/2010/06/basiconfig.jpg" alt="basiconfig" width="705" height="179" /></li>
</ul>
]]></content:encoded>
			<wfw:commentRss>http://blog.endlesscode.com/2010/06/03/python-logging-module/feed/</wfw:commentRss>
		<slash:comments>0</slash:comments>
		</item>
	</channel>
</rss>

