关于缓冲区的一点学习

Start

(pwn)出题的时候没什么特殊需求都会对stdout,stdin设置:

1
2
setvbuf(stdout,0,2,0);
setvbuf(stdin,0,2,0);

但是一直对缓冲区只有个模糊的概念..这次完整地学一遍…

啥是缓冲区

1
缓冲区又称为缓存,它是内存空间的一部分。也就是说,在内存空间中预留了一定的存储空间,这些存储空间用来缓冲输入或输出的数据,这部分预留的空间就叫做缓冲区。

为啥需要缓冲区

  • 计算机对内存的读写速度远超硬盘的读写速度,提高计算机速度可从减少对硬盘的操作次数。
  • 所以可以每次读写的时候多读(写)一些放在缓冲区中如果下次需要再去读可以直接在缓冲区中读取。

总的来说就是提高计算机速度。

缓冲区类型

定义在glibc/libio/iosetvbuf.c

1
2
3
4
5
//https://code.woboq.org/userspace/glibc/libio/iosetvbuf.c.html#_IO_setvbuf

#define _IOFBF 0 /* Fully buffered. */
#define _IOLBF 1 /* Line buffered. */
#define _IONBF 2 /* No buffering. */

一共有三种类型:全缓冲,行缓冲,无缓冲
我们常用的setvbuf(stdout,0,2,0)中的2就是代表无缓冲输入输出直接写入文件
全缓冲的话除了强制刷新缓冲区之外只有缓冲区填满才会刷新。
行缓冲是在全缓冲上增加了遇到换行符也刷新缓冲区。

常见的stdout初始就是行缓冲缓冲区大小为8192bit
glibc/libio/stdio.h中有定义#define BUFSIZ 8192
可以通过以下程序验证。

1
2
3
4
5
6
7
8
#include<stdio.h>
char buf[0x123];
int main()
{
puts("n132");
printf("AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAI");//"A"*0x400+"I"
read(0,buf,0x132);
}

程序会输出1024个”A”但是”I”会留在缓冲区中

输出结束后stdout情况如下

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
n132>>> p *stdout
$1 = {
_flags = 0xfbad2a84,
_IO_read_ptr = 0x602010 "I", 'A' <repeats 14 times>...,
_IO_read_end = 0x602010 "I", 'A' <repeats 14 times>...,
_IO_read_base = 0x602010 "I", 'A' <repeats 14 times>...,
_IO_write_base = 0x602010 "I", 'A' <repeats 14 times>...,
_IO_write_ptr = 0x602011 'A' <repeats 15 times>...,
_IO_write_end = 0x602010 "I", 'A' <repeats 14 times>...,
_IO_buf_base = 0x602010 "I", 'A' <repeats 14 times>...,
_IO_buf_end = 0x602410 "",
_IO_save_base = 0x0,
_IO_backup_base = 0x0,
_IO_save_end = 0x0,
_markers = 0x0,
_chain = 0x7ffff7dd18e0 <_IO_2_1_stdin_>,
_fileno = 0x1,
_flags2 = 0x0,
_old_offset = 0xffffffffffffffff,
_cur_column = 0x0,
_vtable_offset = 0x0,
_shortbuf = "",
_lock = 0x7ffff7dd3780 <_IO_stdfile_1_lock>,
_offset = 0xffffffffffffffff,
_codecvt = 0x0,
_wide_data = 0x7ffff7dd17a0 <_IO_wide_data_1>,
_freeres_list = 0x0,
_freeres_buf = 0x0,
__pad5 = 0x0,
_mode = 0xffffffff,
_unused2 = '\000' <repeats 19 times>
}

这里我们可以发现缓冲区在heap上,一个0x410size的chunk

fllush()

fllush用于清空文件缓冲区

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
//https://code.woboq.org/userspace/glibc/libio/iofflush.c.html
int
_IO_fflush (FILE *fp)
{
if (fp == NULL)
return _IO_flush_all ();
else
{
int result;
CHECK_FILE (fp, EOF);
_IO_acquire_lock (fp);
result = _IO_SYNC (fp) ? EOF : 0;
_IO_release_lock (fp);
return result;
}
}
libc_hidden_def (_IO_fflush)
weak_alias (_IO_fflush, fflush)

setvbuf()

利用setvbuf事实上我们可以设置文件的缓冲模式和缓冲区大小.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
//https://code.woboq.org/userspace/glibc/libio/iosetvbuf.c.html#_IO_setvbuf
int
_IO_setvbuf (FILE *fp, char *buf, int mode, size_t size)
{
int result;
CHECK_FILE (fp, EOF);
_IO_acquire_lock (fp);
switch (mode)
{
case _IOFBF:
fp->_flags &= ~(_IO_LINE_BUF|_IO_UNBUFFERED);
if (buf == NULL)
{
if (fp->_IO_buf_base == NULL)
{
/* There is no flag to distinguish between "fully buffered
mode has been explicitly set" as opposed to "line
buffering has not been explicitly set". In both
cases, _IO_LINE_BUF is off. If this is a tty, and
_IO_filedoalloc later gets called, it cannot know if
it should set the _IO_LINE_BUF flag (because that is
the default), or not (because we have explicitly asked
for fully buffered mode). So we make sure a buffer
gets allocated now, and explicitly turn off line
buffering.
A possibly cleaner alternative would be to add an
extra flag, but then flags are a finite resource. */
if (_IO_DOALLOCATE (fp) < 0)
{
result = EOF;
goto unlock_return;
}
fp->_flags &= ~_IO_LINE_BUF;
}
result = 0;
goto unlock_return;
}
break;
case _IOLBF:
fp->_flags &= ~_IO_UNBUFFERED;
fp->_flags |= _IO_LINE_BUF;
if (buf == NULL)
{
result = 0;
goto unlock_return;
}
break;
case _IONBF:
fp->_flags &= ~_IO_LINE_BUF;
fp->_flags |= _IO_UNBUFFERED;
buf = NULL;
size = 0;
break;
default:
result = EOF;
goto unlock_return;
}
result = _IO_SETBUF (fp, buf, size) == NULL ? EOF : 0;
unlock_return:
_IO_release_lock (fp);
return result;
}
libc_hidden_def (_IO_setvbuf)
weak_alias (_IO_setvbuf, setvbuf)

函数流程:

  • 确定mode
    • 全缓冲
      • 设置_flags
      • 如果buf不为0那么重新调整缓冲区_IO_SETBUF(fp,buf,siez)
      • 如果buf为0,call fp->_IO_doallocate_t,设置_flags
    • 行缓冲
      • 设置_flags
      • 如果buf不为0那么重新调整缓冲区_IO_SETBUF(fp,buf,siez)
    • 无缓冲
      • 设置_flags
      • _IO_SETBUF(fp,0,0)

其中 _IO_SETBUF(fp,buffer_address,size)表示设置fp文件的缓冲区地址为buffer_address大小为size

为了这几种mode_flags先贴上各种_flags宏定义

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
#define _IO_MAGIC         0xFBAD0000 /* Magic number */
#define _IO_MAGIC_MASK 0xFFFF0000
#define _IO_USER_BUF 0x0001 /* Don't deallocate buffer on close. */
#define _IO_UNBUFFERED 0x0002
#define _IO_NO_READS 0x0004 /* Reading not allowed. */
#define _IO_NO_WRITES 0x0008 /* Writing not allowed. */
#define _IO_EOF_SEEN 0x0010
#define _IO_ERR_SEEN 0x0020
#define _IO_DELETE_DONT_CLOSE 0x0040 /* Don't call close(_fileno) on close. */
#define _IO_LINKED 0x0080 /* In the list of all open files. */
#define _IO_IN_BACKUP 0x0100
#define _IO_LINE_BUF 0x0200
#define _IO_TIED_PUT_GET 0x0400 /* Put and get pointer move in unison. */
#define _IO_CURRENTLY_PUTTING 0x0800
#define _IO_IS_APPENDING 0x1000
#define _IO_IS_FILEBUF 0x2000
/* 0x4000 No longer used, reserved for compat. */
#define _IO_USER_LOCK 0x8000

2个涉及到的flag:

  • _IO_LINE_BUF 0x200
  • _IO_UNBUFFERED 0x2

三种case中对_flags的操作依次分别是:

  • 去除_IO_LINE_BUF,_IO_UNBUFFERED
  • 去除_IO_UNBUFFERED增加_IO_LINE_BUF
  • 去除_IO_LINE_BUF增加_IO_UNBUFFERED

tip:感觉看了这个对_IO_LEAK的理解更加深入了
在上一次的探索中得知leak条件是_flags&0x1800=0x1800
其中包括了_IO_LINE_BUF一般ctf比赛中的binary都有设置stdout为无缓冲所以改写是必要的,虽然另一个条件是_IO_IS_APPENDING显然不是stdout具备的.

exit()

这个函数会清空所有fp的缓冲区之后会单独开篇文章研究。

summary

  • 缓冲区的使用为了提高效率
  • 三种缓冲类型,几个对缓冲区操作的函数
    看了挺多资料,基本了解缓冲区工作的原理.