词法分析器_CSS词法分析器

词法分析器

I have so much stuff to do and I've been feeling a little overwhelmed lately. Not depressed, because it's next to impossible to be depressed at a climate including 320 sunny days a year and a beach. So I thought why not drop everything and relax. I'm currently staying at home, enjoying my unused vacation days. So no work, no meetings, no nothing. I thought I should relax by taking on a task that requires some degree of concentration as opposed to just jumping from one task to the next.

我有很多事情要做，最近我有点不知所措。不要沮丧，因为在包括一年320个晴天和海滩的气候下几乎不可能沮丧。所以我想为什么不放弃一切放松自己。我目前待在家里，享受未使用的假期。因此，没有工作，没有会议，什么也没有。我认为我应该放松一些需要集中精力的任务，而不是仅仅从一个任务跳到下一个任务。

I have ideas for a bunch of CSS related tools and utilities, all of which, naturally, require understanding of CSS code. So I need a parser and thought I should write one in JavaScript.

我对一堆与CSS相关的工具和实用程序有一些想法，所有这些自然都需要了解CSS代码。所以我需要一个解析器，并认为我应该用JavaScript编写一个。

The first step is a lexer scanner and I'm happy to share what I committed to github today. It's right here. I called it cssex (yep, cheesy but I didn't want to spend any time thinking of a proper name).

第一步是词法分析器扫描程序，我很乐意分享我今天对github所做的承诺。就在这里。我称它为cssex(是的，俗气，但我不想花任何时间想一个专有名词)。

It's not doing much currently but it's a step. It takes a piece of CSS code and tokenizes it producing the following token types:

它目前没有做很多，但这是一个步骤。它需要一段CSS代码并将其标记化，从而产生以下标记类型：

comment
评论
string
串
white (spaces or tabs)
白色(空格或制表符)
line (new lines)
行(新行)
identifier (could be anything, such as a property or value or font name)
标识符(可以是任何东西，例如属性，值或字体名称)
number
数
match (5 combinations of two operators used in attribute matching such as ^=)
匹配(用于属性匹配(例如^= )的两个运算符的5种组合)
operator - such as . # % * and so on
运算子-例如. # % * . # % *等

You can see a test page here and I'll be happy to hear any bug reports. The test page takes CSS, tokenizes it, then recreates the source from the tokens to compare that the original is reproducible. It also highlights the different tokens in different colors and finally dumps the types and values of the tokens (the complete dump is in console.log)

您可以在此处看到测试页，我们将很高兴听到任何错误报告。测试页使用CSS，对其进行标记，然后根据标记重新创建源，以比较原始源是否可复制。它还以不同的颜色突出显示了不同的令牌，并最终转储令牌的类型和值(完整的转储位于console.log )。

As you can see, I'm continuing with the cheesiness:

如您所见，我在继续俗气：

foreplay.html is the test page (instead of "playground")
foreplay.html是测试页(而不是“操场”)
test-osterone.js (instead of simply "test") is the test runner that uses JavaScriptCore
test-osterone.js (而不是简单的“ test”)是使用JavaScriptCore的测试运行器
penthouse.sh (instead of "suite") runs tests with the 213 CSS files from CSSZenGarden.com
penthouse.sh (而不是“ suite”)使用来自CSSZenGarden.com的213个CSS文件运行测试
sex.js is the lexer itself which defines the global CSSEX object with two methods - lex(source) and its opposite toSource(tokens)
sex.js是词法分析器本身，它使用两种方法定义全局CSSEX对象-lex lex(source)及其与toSource(tokens)相反

So what's next is a proper parser validating those tokens produced by the lexer. Then the tools such as a minifier, highlighter, lint, and whatnot (for example something that will add automatically all -moz- and -o- and stuff to your border-radius). But first I need to draw me some railroad diagrams like those Douglas Crockford has for JavaScript and JSON, they should be immensely helpful when parsing. As you can probably guess, Crockford's JSlint and JSON parser and his writeup on Pratt's top down operator precedence is my source of "view source" 🙂

因此，接下来是一个适当的解析器，用于验证词法分析器生成的那些标记。然后是诸如缩小器，荧光笔，皮棉之类的工具(例如将自动将所有-moz-和-o-以及填充物添加到border-radius东西)的工具。但是首先，我需要绘制一些铁路图，例如Douglas Crockford在JavaScript和JSON中使用的那些，它们在解析时应该非常有用。您可能会猜到，Crockford的JSlint和JSON解析器以及他关于Pratt自上而下的运算符优先级的文章是我“查看源代码”的来源

My main motivation behind all this (other than the itch) is a proper minifier written in JavaScript (therefore running everywhere), not just a collection of regular expressions that YUICSSmin is right now. Also a proper validator, one that understands the nature of the frontend beast and can handle everything from CSS2.1, CSS3's media queries, transitions, latest -webkit and -moz craziness all the way down to IE hacks, expressions, behaviors and filters. And everything in between. Because more often than not we don't validate CSS simply due to the w3c validator being too strict and out of touch with reality.

所有这些背后的主要动机(除了痒)是用JavaScript编写的适当的minifier(因此在各处运行)，而不仅仅是YUICSSmin现在所使用的正则表达式集合。也是一个合适的验证器，可以理解前端野兽的性质，并且可以处理从CSS2.1，CSS3的媒体查询，转换，最新的-webkit和-moz疯狂到IE hack，表达式，行为和过滤器的所有内容。介于两者之间。因为大多数时候我们不仅仅因为w3c验证器太严格并且与现实脱节而不能验证CSS。

Tell your friends about this post on Facebook and Twitter

在Facebook和Twitter上告诉您的朋友有关此帖子的信息